{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "654dc6eb422087a85df50aad9646e3c3b8692e5d"
   },
   "source": [
    "#                                                                                 BERT\n",
    "\n",
    "BERT, or Bidirectional Encoder Representations from Transformers, is a new method of pre-training language representations which obtains state-of-the-art results on a wide array of Natural Language Processing (NLP) tasks.\n",
    "\n",
    "Academic paper which describes BERT in detail and provides full results on a number of tasks can be found here: https://arxiv.org/abs/1810.04805.\n",
    "\n",
    "Github account for the paper can be found here: https://github.com/google-research/bert\n",
    "\n",
    "BERT is a method of pre-training language representations, meaning training of a general-purpose \"language understanding\" model on a large text corpus (like Wikipedia), and then using that model for downstream NLP tasks (like question answering). BERT outperforms previous methods because it is the first *unsupervised, deeply bidirectional *system for pre-training NLP."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "8d6fe072db4c37b40476a41be674be47312d6251"
   },
   "source": [
    "![](https://www.lyrn.ai/wp-content/uploads/2018/11/transformer.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "85bc6b83b518e604a855a149ff9a5ce44f991620"
   },
   "source": [
    "\n",
    "# Downloading all necessary dependencies\n",
    "You will have to turn on internet for that.\n",
    "\n",
    "This code is slightly modefied version of this colab notebook https://colab.research.google.com/github/tensorflow/tpu/blob/master/tools/colab/bert_finetuning_with_cloud_tpus.ipynb"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "_cell_guid": "b1076dfc-b9ad-4769-8c92-a6c4dae69d19",
    "_uuid": "8f2839f25d086af736a60e9eeb907d3b93b6e0e5"
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import os\n",
    "import numpy as np\n",
    "import zipfile\n",
    "from matplotlib import pyplot as plt\n",
    "%matplotlib inline\n",
    "import sys\n",
    "import datetime"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "_cell_guid": "79c7e3d0-c299-4dcb-8224-4455121ee9b0",
    "_uuid": "d629ff2d2480ee46fbb7e2d37f6b5fab8052498a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2019-03-16 11:24:14--  https://storage.googleapis.com/bert_models/2018_10_18/uncased_L-12_H-768_A-12.zip\n",
      "Resolving storage.googleapis.com (storage.googleapis.com)... 74.125.193.128, 2a00:1450:400b:c01::80\n",
      "Connecting to storage.googleapis.com (storage.googleapis.com)|74.125.193.128|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 407727028 (389M) [application/zip]\n",
      "Saving to: ‘uncased_L-12_H-768_A-12.zip’\n",
      "\n",
      "uncased_L-12_H-768_ 100%[===================>] 388.84M   177MB/s    in 2.2s    \n",
      "\n",
      "2019-03-16 11:24:17 (177 MB/s) - ‘uncased_L-12_H-768_A-12.zip’ saved [407727028/407727028]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "#downloading weights and cofiguration file for the model\n",
    "!wget https://storage.googleapis.com/bert_models/2018_10_18/uncased_L-12_H-768_A-12.zip"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "_uuid": "8cf1f5466e25bbf042e0b6d55355d0bf7f02984e"
   },
   "outputs": [],
   "source": [
    "repo = 'model_repo'\n",
    "with zipfile.ZipFile(\"uncased_L-12_H-768_A-12.zip\",\"r\") as zip_ref:\n",
    "    zip_ref.extractall(repo)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "_uuid": "b4a8b81c1fb8b5fae20945a56fda981550c54342"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "bert_config.json\t\t     bert_model.ckpt.index  vocab.txt\r\n",
      "bert_model.ckpt.data-00000-of-00001  bert_model.ckpt.meta\r\n"
     ]
    }
   ],
   "source": [
    "!ls 'model_repo/uncased_L-12_H-768_A-12'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "_uuid": "0595c54d1033b9b0e69c5e565a8063a32295c3ff"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2019-03-12 10:25:25--  https://raw.githubusercontent.com/google-research/bert/master/modeling.py\n",
      "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.16.133\n",
      "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.16.133|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 37922 (37K) [text/plain]\n",
      "Saving to: ‘modeling.py’\n",
      "\n",
      "modeling.py         100%[===================>]  37.03K  --.-KB/s    in 0.01s   \n",
      "\n",
      "2019-03-12 10:25:25 (3.48 MB/s) - ‘modeling.py’ saved [37922/37922]\n",
      "\n",
      "--2019-03-12 10:25:26--  https://raw.githubusercontent.com/google-research/bert/master/optimization.py\n",
      "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.16.133\n",
      "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.16.133|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 6258 (6.1K) [text/plain]\n",
      "Saving to: ‘optimization.py’\n",
      "\n",
      "optimization.py     100%[===================>]   6.11K  --.-KB/s    in 0s      \n",
      "\n",
      "2019-03-12 10:25:26 (136 MB/s) - ‘optimization.py’ saved [6258/6258]\n",
      "\n",
      "--2019-03-12 10:25:26--  https://raw.githubusercontent.com/google-research/bert/master/run_classifier.py\n",
      "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.16.133\n",
      "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.16.133|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 34783 (34K) [text/plain]\n",
      "Saving to: ‘run_classifier.py’\n",
      "\n",
      "run_classifier.py   100%[===================>]  33.97K  --.-KB/s    in 0.01s   \n",
      "\n",
      "2019-03-12 10:25:26 (3.21 MB/s) - ‘run_classifier.py’ saved [34783/34783]\n",
      "\n",
      "--2019-03-12 10:25:26--  https://raw.githubusercontent.com/google-research/bert/master/tokenization.py\n",
      "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.16.133\n",
      "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.16.133|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 12257 (12K) [text/plain]\n",
      "Saving to: ‘tokenization.py’\n",
      "\n",
      "tokenization.py     100%[===================>]  11.97K  --.-KB/s    in 0s      \n",
      "\n",
      "2019-03-12 10:25:27 (178 MB/s) - ‘tokenization.py’ saved [12257/12257]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!wget https://raw.githubusercontent.com/google-research/bert/master/modeling.py \n",
    "!wget https://raw.githubusercontent.com/google-research/bert/master/optimization.py \n",
    "!wget https://raw.githubusercontent.com/google-research/bert/master/run_classifier.py \n",
    "!wget https://raw.githubusercontent.com/google-research/bert/master/tokenization.py "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "68f9932a52af968d1e94d0e115fc06e3423e1d47"
   },
   "source": [
    "Example below is done on preprocessing code, similar to **CoLa**:\n",
    "\n",
    "The Corpus of Linguistic Acceptability is\n",
    "a binary single-sentence classification task, where \n",
    "the goal is to predict whether an English sentence\n",
    "is linguistically “acceptable” or not\n",
    "\n",
    "You can use pretrained BERT model for wide variety of tasks, including classification.\n",
    "The task of CoLa is close to the task of Quora competition, so I thought it woud be interesting to use that example.\n",
    "Obviously, outside sources aren't allowed in Quora competition, so you won't be able to use BERT to submit a prediction.\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "_uuid": "0f47047973d523029fbe9c355577133c4656ce57"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "***** Model output directory: model_repo/outputs *****\n",
      "***** BERT pretrained directory: model_repo/uncased_L-12_H-768_A-12 *****\n"
     ]
    }
   ],
   "source": [
    "# Available pretrained model checkpoints:\n",
    "#   uncased_L-12_H-768_A-12: uncased BERT base model\n",
    "#   uncased_L-24_H-1024_A-16: uncased BERT large model\n",
    "#   cased_L-12_H-768_A-12: cased BERT large model\n",
    "#We will use the most basic of all of them\n",
    "BERT_MODEL = 'uncased_L-12_H-768_A-12'\n",
    "BERT_PRETRAINED_DIR = f'{repo}/uncased_L-12_H-768_A-12'\n",
    "OUTPUT_DIR = f'{repo}/outputs'\n",
    "print(f'***** Model output directory: {OUTPUT_DIR} *****')\n",
    "print(f'***** BERT pretrained directory: {BERT_PRETRAINED_DIR} *****')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "_uuid": "3a2a0d12de6ecb82812a36549ea910b5a55c0c4a"
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "train_df =  pd.read_csv('input/train.csv')\n",
    "\n",
    "train, test = train_test_split(train_df, test_size = 0.1, random_state=42)\n",
    "\n",
    "train_lines, train_labels = train.question_text.values, train.target.values\n",
    "test_lines, test_labels = test.question_text.values, test.target.values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "checkpoint\r\n",
      "events.out.tfevents.1552492994.TestPri2\r\n",
      "events.out.tfevents.1552506308.TestPri2\r\n",
      "events.out.tfevents.1552509591.TestPri2\r\n",
      "events.out.tfevents.1552515231.TestPri2\r\n",
      "events.out.tfevents.1552576467.TestPri2\r\n",
      "events.out.tfevents.1552577887.TestPri2\r\n",
      "events.out.tfevents.1552586048.TestPri2\r\n",
      "events.out.tfevents.1552599670.TestPri2\r\n",
      "events.out.tfevents.1552608533.TestPri2\r\n",
      "graph.pbtxt\r\n",
      "graph.pbtxt.tmp7905681f66134ab8aef12e8b07bddc9f\r\n",
      "model.ckpt-88000.data-00000-of-00001\r\n",
      "model.ckpt-88000.index\r\n",
      "model.ckpt-88000.meta\r\n",
      "model.ckpt-89000.data-00000-of-00001\r\n",
      "model.ckpt-89000.index\r\n",
      "model.ckpt-89000.meta\r\n",
      "model.ckpt-90000.data-00000-of-00001\r\n",
      "model.ckpt-90000.index\r\n",
      "model.ckpt-90000.meta\r\n",
      "model.ckpt-91000.data-00000-of-00001\r\n",
      "model.ckpt-91000.index\r\n",
      "model.ckpt-91000.meta\r\n",
      "model.ckpt-91836.data-00000-of-00001\r\n",
      "model.ckpt-91836.index\r\n",
      "model.ckpt-91836.meta\r\n"
     ]
    }
   ],
   "source": [
    "! ls model_repo/outputs/\n",
    "#! rm -r model_repo/outputs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "_uuid": "b8804d16e48636d8a5c1474c4384076b16c73adc"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:Estimator's model_fn (<function model_fn_builder.<locals>.model_fn at 0x7fae945c4c80>) includes params argument, but params are not passed to Estimator.\n",
      "INFO:tensorflow:Using config: {'_model_dir': 'model_repo/outputs', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': 1000, '_save_checkpoints_secs': None, '_session_config': allow_soft_placement: true\n",
      "graph_options {\n",
      "  rewrite_options {\n",
      "    meta_optimizer_iterations: ONE\n",
      "  }\n",
      "}\n",
      ", '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': None, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_service': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7faeaa1c5780>, '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1, '_tpu_config': TPUConfig(iterations_per_loop=1000, num_shards=8, num_cores_per_replica=None, per_host_input_for_training=3, tpu_job_name=None, initial_infeed_sleep_secs=None, input_partition_dims=None), '_cluster': None}\n",
      "INFO:tensorflow:_TPUContext: eval_on_tpu True\n",
      "WARNING:tensorflow:eval_on_tpu ignored because use_tpu is False.\n"
     ]
    }
   ],
   "source": [
    "import modeling\n",
    "import optimization\n",
    "import run_classifier\n",
    "import tokenization\n",
    "import tensorflow as tf\n",
    "\n",
    "\n",
    "def create_examples(lines, set_type, labels=None):\n",
    "#Generate data for the BERT model\n",
    "    guid = f'{set_type}'\n",
    "    examples = []\n",
    "    if guid == 'train':\n",
    "        for line, label in zip(lines, labels):\n",
    "            text_a = line\n",
    "            label = str(label)\n",
    "            examples.append(\n",
    "              run_classifier.InputExample(guid=guid, text_a=text_a, text_b=None, label=label))\n",
    "    else:\n",
    "        for line in lines:\n",
    "            text_a = line\n",
    "            label = '0'\n",
    "            examples.append(\n",
    "              run_classifier.InputExample(guid=guid, text_a=text_a, text_b=None, label=label))\n",
    "    return examples\n",
    "\n",
    "TRAIN_BATCH_SIZE = 32\n",
    "EVAL_BATCH_SIZE = 8\n",
    "LEARNING_RATE = 2e-5\n",
    "NUM_TRAIN_EPOCHS = 2.5\n",
    "WARMUP_PROPORTION = 0.1\n",
    "MAX_SEQ_LENGTH = 128\n",
    "# Model configs\n",
    "SAVE_CHECKPOINTS_STEPS = 1000 #if you wish to finetune a model on a larger dataset, use larger interval\n",
    "# each checpoint weights about 1,5gb\n",
    "ITERATIONS_PER_LOOP = 1000\n",
    "NUM_TPU_CORES = 8\n",
    "VOCAB_FILE = os.path.join(BERT_PRETRAINED_DIR, 'vocab.txt')\n",
    "CONFIG_FILE = os.path.join(BERT_PRETRAINED_DIR, 'bert_config.json')\n",
    "INIT_CHECKPOINT = os.path.join(OUTPUT_DIR, 'model.ckpt-91836')\n",
    "DO_LOWER_CASE = BERT_MODEL.startswith('uncased')\n",
    "\n",
    "label_list = ['0', '1']\n",
    "tokenizer = tokenization.FullTokenizer(vocab_file=VOCAB_FILE, do_lower_case=DO_LOWER_CASE)\n",
    "train_examples = create_examples(train_lines, 'train', labels=train_labels)\n",
    "\n",
    "tpu_cluster_resolver = None #Since training will happen on GPU, we won't need a cluster resolver\n",
    "#TPUEstimator also supports training on CPU and GPU. You don't need to define a separate tf.estimator.Estimator.\n",
    "run_config = tf.contrib.tpu.RunConfig(\n",
    "    cluster=tpu_cluster_resolver,\n",
    "    model_dir=OUTPUT_DIR,\n",
    "    save_checkpoints_steps=SAVE_CHECKPOINTS_STEPS,\n",
    "    tpu_config=tf.contrib.tpu.TPUConfig(\n",
    "        iterations_per_loop=ITERATIONS_PER_LOOP,\n",
    "        num_shards=NUM_TPU_CORES,\n",
    "        per_host_input_for_training=tf.contrib.tpu.InputPipelineConfig.PER_HOST_V2))\n",
    "\n",
    "num_train_steps = int(\n",
    "    len(train_examples) / TRAIN_BATCH_SIZE * NUM_TRAIN_EPOCHS)\n",
    "num_warmup_steps = int(num_train_steps * WARMUP_PROPORTION)\n",
    "\n",
    "model_fn = run_classifier.model_fn_builder(\n",
    "    bert_config=modeling.BertConfig.from_json_file(CONFIG_FILE),\n",
    "    num_labels=len(label_list),\n",
    "    init_checkpoint=INIT_CHECKPOINT,\n",
    "    learning_rate=LEARNING_RATE,\n",
    "    num_train_steps=num_train_steps,\n",
    "    num_warmup_steps=num_warmup_steps,\n",
    "    use_tpu=False, #If False training will fall on CPU or GPU, depending on what is available  \n",
    "    use_one_hot_embeddings=True)\n",
    "\n",
    "estimator = tf.contrib.tpu.TPUEstimator(\n",
    "    use_tpu=False, #If False training will fall on CPU or GPU, depending on what is available \n",
    "    model_fn=model_fn,\n",
    "    config=run_config,\n",
    "    train_batch_size=TRAIN_BATCH_SIZE,\n",
    "    eval_batch_size=EVAL_BATCH_SIZE)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "_uuid": "b69a1b734026d0b4b16c3eeb405e920195183873"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Please wait...\n",
      "INFO:tensorflow:Writing example 0 of 1175509\n",
      "INFO:tensorflow:*** Example ***\n",
      "INFO:tensorflow:guid: train\n",
      "INFO:tensorflow:tokens: [CLS] are the candidates given a holiday after the 6 months training at indian naval academy ? [SEP]\n",
      "INFO:tensorflow:input_ids: 101 2024 1996 5347 2445 1037 6209 2044 1996 1020 2706 2731 2012 2796 3987 2914 1029 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
      "INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
      "INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
      "INFO:tensorflow:label: 0 (id = 0)\n",
      "INFO:tensorflow:*** Example ***\n",
      "INFO:tensorflow:guid: train\n",
      "INFO:tensorflow:tokens: [CLS] why is robb ##en island a tourist attraction ? [SEP]\n",
      "INFO:tensorflow:input_ids: 101 2339 2003 26211 2368 2479 1037 7538 8432 1029 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
      "INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
      "INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
      "INFO:tensorflow:label: 0 (id = 0)\n",
      "INFO:tensorflow:*** Example ***\n",
      "INFO:tensorflow:guid: train\n",
      "INFO:tensorflow:tokens: [CLS] how can you identify the different themes addressed in the novel \" the visible ##s \" by sara shepard ? [SEP]\n",
      "INFO:tensorflow:input_ids: 101 2129 2064 2017 6709 1996 2367 6991 8280 1999 1996 3117 1000 1996 5710 2015 1000 2011 7354 22189 1029 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
      "INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
      "INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
      "INFO:tensorflow:label: 0 (id = 0)\n",
      "INFO:tensorflow:*** Example ***\n",
      "INFO:tensorflow:guid: train\n",
      "INFO:tensorflow:tokens: [CLS] why are believers against homosexuality though everyone is gay ? [SEP]\n",
      "INFO:tensorflow:input_ids: 101 2339 2024 20373 2114 15949 2295 3071 2003 5637 1029 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
      "INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
      "INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
      "INFO:tensorflow:label: 0 (id = 0)\n",
      "INFO:tensorflow:*** Example ***\n",
      "INFO:tensorflow:guid: train\n",
      "INFO:tensorflow:tokens: [CLS] how would marcos perform against naval commandos of p ##5 ? [SEP]\n",
      "INFO:tensorflow:input_ids: 101 2129 2052 14810 4685 2114 3987 25144 1997 1052 2629 1029 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
      "INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
      "INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
      "INFO:tensorflow:label: 0 (id = 0)\n",
      "INFO:tensorflow:Writing example 10000 of 1175509\n",
      "INFO:tensorflow:Writing example 20000 of 1175509\n",
      "INFO:tensorflow:Writing example 30000 of 1175509\n",
      "INFO:tensorflow:Writing example 40000 of 1175509\n",
      "INFO:tensorflow:Writing example 50000 of 1175509\n",
      "INFO:tensorflow:Writing example 60000 of 1175509\n",
      "INFO:tensorflow:Writing example 70000 of 1175509\n",
      "INFO:tensorflow:Writing example 80000 of 1175509\n",
      "INFO:tensorflow:Writing example 90000 of 1175509\n",
      "INFO:tensorflow:Writing example 100000 of 1175509\n",
      "INFO:tensorflow:Writing example 110000 of 1175509\n",
      "INFO:tensorflow:Writing example 120000 of 1175509\n",
      "INFO:tensorflow:Writing example 130000 of 1175509\n",
      "INFO:tensorflow:Writing example 140000 of 1175509\n",
      "INFO:tensorflow:Writing example 150000 of 1175509\n",
      "INFO:tensorflow:Writing example 160000 of 1175509\n",
      "INFO:tensorflow:Writing example 170000 of 1175509\n",
      "INFO:tensorflow:Writing example 180000 of 1175509\n",
      "INFO:tensorflow:Writing example 190000 of 1175509\n",
      "INFO:tensorflow:Writing example 200000 of 1175509\n",
      "INFO:tensorflow:Writing example 210000 of 1175509\n",
      "INFO:tensorflow:Writing example 220000 of 1175509\n",
      "INFO:tensorflow:Writing example 230000 of 1175509\n",
      "INFO:tensorflow:Writing example 240000 of 1175509\n",
      "INFO:tensorflow:Writing example 250000 of 1175509\n",
      "INFO:tensorflow:Writing example 260000 of 1175509\n",
      "INFO:tensorflow:Writing example 270000 of 1175509\n",
      "INFO:tensorflow:Writing example 280000 of 1175509\n",
      "INFO:tensorflow:Writing example 290000 of 1175509\n",
      "INFO:tensorflow:Writing example 300000 of 1175509\n",
      "INFO:tensorflow:Writing example 310000 of 1175509\n",
      "INFO:tensorflow:Writing example 320000 of 1175509\n",
      "INFO:tensorflow:Writing example 330000 of 1175509\n",
      "INFO:tensorflow:Writing example 340000 of 1175509\n",
      "INFO:tensorflow:Writing example 350000 of 1175509\n",
      "INFO:tensorflow:Writing example 360000 of 1175509\n",
      "INFO:tensorflow:Writing example 370000 of 1175509\n",
      "INFO:tensorflow:Writing example 380000 of 1175509\n",
      "INFO:tensorflow:Writing example 390000 of 1175509\n",
      "INFO:tensorflow:Writing example 400000 of 1175509\n",
      "INFO:tensorflow:Writing example 410000 of 1175509\n",
      "INFO:tensorflow:Writing example 420000 of 1175509\n",
      "INFO:tensorflow:Writing example 430000 of 1175509\n",
      "INFO:tensorflow:Writing example 440000 of 1175509\n",
      "INFO:tensorflow:Writing example 450000 of 1175509\n",
      "INFO:tensorflow:Writing example 460000 of 1175509\n",
      "INFO:tensorflow:Writing example 470000 of 1175509\n",
      "INFO:tensorflow:Writing example 480000 of 1175509\n",
      "INFO:tensorflow:Writing example 490000 of 1175509\n",
      "INFO:tensorflow:Writing example 500000 of 1175509\n",
      "INFO:tensorflow:Writing example 510000 of 1175509\n",
      "INFO:tensorflow:Writing example 520000 of 1175509\n",
      "INFO:tensorflow:Writing example 530000 of 1175509\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Writing example 540000 of 1175509\n",
      "INFO:tensorflow:Writing example 550000 of 1175509\n",
      "INFO:tensorflow:Writing example 560000 of 1175509\n",
      "INFO:tensorflow:Writing example 570000 of 1175509\n",
      "INFO:tensorflow:Writing example 580000 of 1175509\n",
      "INFO:tensorflow:Writing example 590000 of 1175509\n",
      "INFO:tensorflow:Writing example 600000 of 1175509\n",
      "INFO:tensorflow:Writing example 610000 of 1175509\n",
      "INFO:tensorflow:Writing example 620000 of 1175509\n",
      "INFO:tensorflow:Writing example 630000 of 1175509\n",
      "INFO:tensorflow:Writing example 640000 of 1175509\n",
      "INFO:tensorflow:Writing example 650000 of 1175509\n",
      "INFO:tensorflow:Writing example 660000 of 1175509\n",
      "INFO:tensorflow:Writing example 670000 of 1175509\n",
      "INFO:tensorflow:Writing example 680000 of 1175509\n",
      "INFO:tensorflow:Writing example 690000 of 1175509\n",
      "INFO:tensorflow:Writing example 700000 of 1175509\n",
      "INFO:tensorflow:Writing example 710000 of 1175509\n",
      "INFO:tensorflow:Writing example 720000 of 1175509\n",
      "INFO:tensorflow:Writing example 730000 of 1175509\n",
      "INFO:tensorflow:Writing example 740000 of 1175509\n",
      "INFO:tensorflow:Writing example 750000 of 1175509\n",
      "INFO:tensorflow:Writing example 760000 of 1175509\n",
      "INFO:tensorflow:Writing example 770000 of 1175509\n",
      "INFO:tensorflow:Writing example 780000 of 1175509\n",
      "INFO:tensorflow:Writing example 790000 of 1175509\n",
      "INFO:tensorflow:Writing example 800000 of 1175509\n",
      "INFO:tensorflow:Writing example 810000 of 1175509\n",
      "INFO:tensorflow:Writing example 820000 of 1175509\n",
      "INFO:tensorflow:Writing example 830000 of 1175509\n",
      "INFO:tensorflow:Writing example 840000 of 1175509\n",
      "INFO:tensorflow:Writing example 850000 of 1175509\n",
      "INFO:tensorflow:Writing example 860000 of 1175509\n",
      "INFO:tensorflow:Writing example 870000 of 1175509\n",
      "INFO:tensorflow:Writing example 880000 of 1175509\n",
      "INFO:tensorflow:Writing example 890000 of 1175509\n",
      "INFO:tensorflow:Writing example 900000 of 1175509\n",
      "INFO:tensorflow:Writing example 910000 of 1175509\n",
      "INFO:tensorflow:Writing example 920000 of 1175509\n",
      "INFO:tensorflow:Writing example 930000 of 1175509\n",
      "INFO:tensorflow:Writing example 940000 of 1175509\n",
      "INFO:tensorflow:Writing example 950000 of 1175509\n",
      "INFO:tensorflow:Writing example 960000 of 1175509\n",
      "INFO:tensorflow:Writing example 970000 of 1175509\n",
      "INFO:tensorflow:Writing example 980000 of 1175509\n",
      "INFO:tensorflow:Writing example 990000 of 1175509\n",
      "INFO:tensorflow:Writing example 1000000 of 1175509\n",
      "INFO:tensorflow:Writing example 1010000 of 1175509\n",
      "INFO:tensorflow:Writing example 1020000 of 1175509\n",
      "INFO:tensorflow:Writing example 1030000 of 1175509\n",
      "INFO:tensorflow:Writing example 1040000 of 1175509\n",
      "INFO:tensorflow:Writing example 1050000 of 1175509\n",
      "INFO:tensorflow:Writing example 1060000 of 1175509\n",
      "INFO:tensorflow:Writing example 1070000 of 1175509\n",
      "INFO:tensorflow:Writing example 1080000 of 1175509\n",
      "INFO:tensorflow:Writing example 1090000 of 1175509\n",
      "INFO:tensorflow:Writing example 1100000 of 1175509\n",
      "INFO:tensorflow:Writing example 1110000 of 1175509\n",
      "INFO:tensorflow:Writing example 1120000 of 1175509\n",
      "INFO:tensorflow:Writing example 1130000 of 1175509\n",
      "INFO:tensorflow:Writing example 1140000 of 1175509\n",
      "INFO:tensorflow:Writing example 1150000 of 1175509\n",
      "INFO:tensorflow:Writing example 1160000 of 1175509\n",
      "INFO:tensorflow:Writing example 1170000 of 1175509\n",
      "***** Started training at 2019-03-14 23:59:03.671111 *****\n",
      "  Num examples = 1175509\n",
      "  Batch size = 32\n",
      "INFO:tensorflow:  Num steps = 91836\n",
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Running train on CPU\n",
      "INFO:tensorflow:*** Features ***\n",
      "INFO:tensorflow:  name = input_ids, shape = (32, 128)\n",
      "INFO:tensorflow:  name = input_mask, shape = (32, 128)\n",
      "INFO:tensorflow:  name = label_ids, shape = (32,)\n",
      "INFO:tensorflow:  name = segment_ids, shape = (32, 128)\n",
      "INFO:tensorflow:**** Trainable Variables ****\n",
      "INFO:tensorflow:  name = bert/embeddings/word_embeddings:0, shape = (30522, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/embeddings/token_type_embeddings:0, shape = (2, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/embeddings/position_embeddings:0, shape = (512, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/embeddings/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/embeddings/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_0/attention/self/query/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_0/attention/self/query/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_0/attention/self/key/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_0/attention/self/key/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_0/attention/self/value/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_0/attention/self/value/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_0/attention/output/dense/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_0/attention/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_0/attention/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_0/attention/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_0/intermediate/dense/kernel:0, shape = (768, 3072), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_0/intermediate/dense/bias:0, shape = (3072,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_0/output/dense/kernel:0, shape = (3072, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_0/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_0/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_0/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_1/attention/self/query/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_1/attention/self/query/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_1/attention/self/key/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_1/attention/self/key/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_1/attention/self/value/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_1/attention/self/value/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_1/attention/output/dense/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_1/attention/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_1/attention/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_1/attention/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_1/intermediate/dense/kernel:0, shape = (768, 3072), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_1/intermediate/dense/bias:0, shape = (3072,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_1/output/dense/kernel:0, shape = (3072, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_1/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_1/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_1/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_2/attention/self/query/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_2/attention/self/query/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_2/attention/self/key/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_2/attention/self/key/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_2/attention/self/value/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:  name = bert/encoder/layer_2/attention/self/value/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_2/attention/output/dense/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_2/attention/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_2/attention/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_2/attention/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_2/intermediate/dense/kernel:0, shape = (768, 3072), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_2/intermediate/dense/bias:0, shape = (3072,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_2/output/dense/kernel:0, shape = (3072, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_2/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_2/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_2/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_3/attention/self/query/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_3/attention/self/query/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_3/attention/self/key/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_3/attention/self/key/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_3/attention/self/value/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_3/attention/self/value/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_3/attention/output/dense/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_3/attention/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_3/attention/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_3/attention/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_3/intermediate/dense/kernel:0, shape = (768, 3072), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_3/intermediate/dense/bias:0, shape = (3072,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_3/output/dense/kernel:0, shape = (3072, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_3/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_3/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_3/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_4/attention/self/query/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_4/attention/self/query/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_4/attention/self/key/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_4/attention/self/key/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_4/attention/self/value/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_4/attention/self/value/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_4/attention/output/dense/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_4/attention/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_4/attention/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_4/attention/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_4/intermediate/dense/kernel:0, shape = (768, 3072), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_4/intermediate/dense/bias:0, shape = (3072,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_4/output/dense/kernel:0, shape = (3072, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_4/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_4/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_4/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_5/attention/self/query/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_5/attention/self/query/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_5/attention/self/key/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_5/attention/self/key/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_5/attention/self/value/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_5/attention/self/value/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_5/attention/output/dense/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_5/attention/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_5/attention/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_5/attention/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_5/intermediate/dense/kernel:0, shape = (768, 3072), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_5/intermediate/dense/bias:0, shape = (3072,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_5/output/dense/kernel:0, shape = (3072, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_5/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_5/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_5/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_6/attention/self/query/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_6/attention/self/query/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_6/attention/self/key/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_6/attention/self/key/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_6/attention/self/value/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_6/attention/self/value/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_6/attention/output/dense/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_6/attention/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_6/attention/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_6/attention/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_6/intermediate/dense/kernel:0, shape = (768, 3072), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_6/intermediate/dense/bias:0, shape = (3072,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_6/output/dense/kernel:0, shape = (3072, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_6/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_6/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_6/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:  name = bert/encoder/layer_7/attention/self/query/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_7/attention/self/query/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_7/attention/self/key/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_7/attention/self/key/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_7/attention/self/value/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_7/attention/self/value/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_7/attention/output/dense/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_7/attention/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_7/attention/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_7/attention/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_7/intermediate/dense/kernel:0, shape = (768, 3072), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_7/intermediate/dense/bias:0, shape = (3072,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_7/output/dense/kernel:0, shape = (3072, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_7/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_7/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_7/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_8/attention/self/query/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_8/attention/self/query/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_8/attention/self/key/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_8/attention/self/key/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_8/attention/self/value/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_8/attention/self/value/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_8/attention/output/dense/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_8/attention/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_8/attention/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_8/attention/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_8/intermediate/dense/kernel:0, shape = (768, 3072), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_8/intermediate/dense/bias:0, shape = (3072,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_8/output/dense/kernel:0, shape = (3072, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_8/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_8/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_8/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_9/attention/self/query/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_9/attention/self/query/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_9/attention/self/key/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_9/attention/self/key/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_9/attention/self/value/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_9/attention/self/value/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_9/attention/output/dense/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_9/attention/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_9/attention/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_9/attention/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_9/intermediate/dense/kernel:0, shape = (768, 3072), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_9/intermediate/dense/bias:0, shape = (3072,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_9/output/dense/kernel:0, shape = (3072, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_9/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_9/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_9/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_10/attention/self/query/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_10/attention/self/query/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_10/attention/self/key/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_10/attention/self/key/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_10/attention/self/value/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_10/attention/self/value/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_10/attention/output/dense/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_10/attention/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_10/attention/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_10/attention/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_10/intermediate/dense/kernel:0, shape = (768, 3072), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_10/intermediate/dense/bias:0, shape = (3072,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_10/output/dense/kernel:0, shape = (3072, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_10/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_10/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_10/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_11/attention/self/query/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_11/attention/self/query/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_11/attention/self/key/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_11/attention/self/key/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_11/attention/self/value/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_11/attention/self/value/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_11/attention/output/dense/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_11/attention/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_11/attention/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_11/attention/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_11/intermediate/dense/kernel:0, shape = (768, 3072), *INIT_FROM_CKPT*\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:  name = bert/encoder/layer_11/intermediate/dense/bias:0, shape = (3072,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_11/output/dense/kernel:0, shape = (3072, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_11/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_11/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_11/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/pooler/dense/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/pooler/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = output_weights:0, shape = (2, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = output_bias:0, shape = (2,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Create CheckpointSaverHook.\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Restoring parameters from model_repo/outputs/model.ckpt-29000\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n",
      "INFO:tensorflow:Saving checkpoints for 29000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.571922\n",
      "INFO:tensorflow:examples/sec: 18.3015\n",
      "INFO:tensorflow:global_step/sec: 0.585601\n",
      "INFO:tensorflow:examples/sec: 18.7392\n",
      "INFO:tensorflow:global_step/sec: 0.582812\n",
      "INFO:tensorflow:examples/sec: 18.65\n",
      "INFO:tensorflow:global_step/sec: 0.581493\n",
      "INFO:tensorflow:examples/sec: 18.6078\n",
      "INFO:tensorflow:global_step/sec: 0.581162\n",
      "INFO:tensorflow:examples/sec: 18.5972\n",
      "INFO:tensorflow:global_step/sec: 0.58126\n",
      "INFO:tensorflow:examples/sec: 18.6003\n",
      "INFO:tensorflow:global_step/sec: 0.581484\n",
      "INFO:tensorflow:examples/sec: 18.6075\n",
      "INFO:tensorflow:global_step/sec: 0.580439\n",
      "INFO:tensorflow:examples/sec: 18.574\n",
      "INFO:tensorflow:global_step/sec: 0.580739\n",
      "INFO:tensorflow:examples/sec: 18.5836\n",
      "INFO:tensorflow:Saving checkpoints for 30000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.543976\n",
      "INFO:tensorflow:examples/sec: 17.4072\n",
      "INFO:tensorflow:global_step/sec: 0.583118\n",
      "INFO:tensorflow:examples/sec: 18.6598\n",
      "INFO:tensorflow:global_step/sec: 0.581773\n",
      "INFO:tensorflow:examples/sec: 18.6167\n",
      "INFO:tensorflow:global_step/sec: 0.581094\n",
      "INFO:tensorflow:examples/sec: 18.595\n",
      "INFO:tensorflow:global_step/sec: 0.5813\n",
      "INFO:tensorflow:examples/sec: 18.6016\n",
      "INFO:tensorflow:global_step/sec: 0.581423\n",
      "INFO:tensorflow:examples/sec: 18.6055\n",
      "INFO:tensorflow:global_step/sec: 0.580421\n",
      "INFO:tensorflow:examples/sec: 18.5735\n",
      "INFO:tensorflow:global_step/sec: 0.580639\n",
      "INFO:tensorflow:examples/sec: 18.5804\n",
      "INFO:tensorflow:global_step/sec: 0.581366\n",
      "INFO:tensorflow:examples/sec: 18.6037\n",
      "INFO:tensorflow:global_step/sec: 0.581671\n",
      "INFO:tensorflow:examples/sec: 18.6135\n",
      "INFO:tensorflow:Saving checkpoints for 31000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.544261\n",
      "INFO:tensorflow:examples/sec: 17.4163\n",
      "INFO:tensorflow:global_step/sec: 0.582802\n",
      "INFO:tensorflow:examples/sec: 18.6496\n",
      "INFO:tensorflow:global_step/sec: 0.580977\n",
      "INFO:tensorflow:examples/sec: 18.5912\n",
      "INFO:tensorflow:global_step/sec: 0.581909\n",
      "INFO:tensorflow:examples/sec: 18.6211\n",
      "INFO:tensorflow:global_step/sec: 0.580479\n",
      "INFO:tensorflow:examples/sec: 18.5753\n",
      "INFO:tensorflow:global_step/sec: 0.581182\n",
      "INFO:tensorflow:examples/sec: 18.5978\n",
      "INFO:tensorflow:global_step/sec: 0.580856\n",
      "INFO:tensorflow:examples/sec: 18.5874\n",
      "INFO:tensorflow:global_step/sec: 0.580817\n",
      "INFO:tensorflow:examples/sec: 18.5861\n",
      "INFO:tensorflow:global_step/sec: 0.5818\n",
      "INFO:tensorflow:examples/sec: 18.6176\n",
      "INFO:tensorflow:global_step/sec: 0.581292\n",
      "INFO:tensorflow:examples/sec: 18.6013\n",
      "INFO:tensorflow:Saving checkpoints for 32000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.544221\n",
      "INFO:tensorflow:examples/sec: 17.4151\n",
      "INFO:tensorflow:global_step/sec: 0.582607\n",
      "INFO:tensorflow:examples/sec: 18.6434\n",
      "INFO:tensorflow:global_step/sec: 0.58253\n",
      "INFO:tensorflow:examples/sec: 18.641\n",
      "INFO:tensorflow:global_step/sec: 0.581735\n",
      "INFO:tensorflow:examples/sec: 18.6155\n",
      "INFO:tensorflow:global_step/sec: 0.581138\n",
      "INFO:tensorflow:examples/sec: 18.5964\n",
      "INFO:tensorflow:global_step/sec: 0.580957\n",
      "INFO:tensorflow:examples/sec: 18.5906\n",
      "INFO:tensorflow:global_step/sec: 0.581137\n",
      "INFO:tensorflow:examples/sec: 18.5964\n",
      "INFO:tensorflow:global_step/sec: 0.580287\n",
      "INFO:tensorflow:examples/sec: 18.5692\n",
      "INFO:tensorflow:global_step/sec: 0.581601\n",
      "INFO:tensorflow:examples/sec: 18.6112\n",
      "INFO:tensorflow:global_step/sec: 0.581015\n",
      "INFO:tensorflow:examples/sec: 18.5925\n",
      "INFO:tensorflow:Saving checkpoints for 33000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.513996\n",
      "INFO:tensorflow:examples/sec: 16.4479\n",
      "INFO:tensorflow:global_step/sec: 0.584232\n",
      "INFO:tensorflow:examples/sec: 18.6954\n",
      "INFO:tensorflow:global_step/sec: 0.582253\n",
      "INFO:tensorflow:examples/sec: 18.6321\n",
      "INFO:tensorflow:global_step/sec: 0.580604\n",
      "INFO:tensorflow:examples/sec: 18.5793\n",
      "INFO:tensorflow:global_step/sec: 0.580659\n",
      "INFO:tensorflow:examples/sec: 18.5811\n",
      "INFO:tensorflow:global_step/sec: 0.581024\n",
      "INFO:tensorflow:examples/sec: 18.5928\n",
      "INFO:tensorflow:global_step/sec: 0.581093\n",
      "INFO:tensorflow:examples/sec: 18.595\n",
      "INFO:tensorflow:global_step/sec: 0.581365\n",
      "INFO:tensorflow:examples/sec: 18.6037\n",
      "INFO:tensorflow:global_step/sec: 0.580572\n",
      "INFO:tensorflow:examples/sec: 18.5783\n",
      "INFO:tensorflow:global_step/sec: 0.581292\n",
      "INFO:tensorflow:examples/sec: 18.6013\n",
      "INFO:tensorflow:Saving checkpoints for 34000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.542766\n",
      "INFO:tensorflow:examples/sec: 17.3685\n",
      "INFO:tensorflow:global_step/sec: 0.583521\n",
      "INFO:tensorflow:examples/sec: 18.6727\n",
      "INFO:tensorflow:global_step/sec: 0.582052\n",
      "INFO:tensorflow:examples/sec: 18.6257\n",
      "INFO:tensorflow:global_step/sec: 0.581766\n",
      "INFO:tensorflow:examples/sec: 18.6165\n",
      "INFO:tensorflow:global_step/sec: 0.58186\n",
      "INFO:tensorflow:examples/sec: 18.6195\n",
      "INFO:tensorflow:global_step/sec: 0.580857\n",
      "INFO:tensorflow:examples/sec: 18.5874\n",
      "INFO:tensorflow:global_step/sec: 0.580499\n",
      "INFO:tensorflow:examples/sec: 18.576\n",
      "INFO:tensorflow:global_step/sec: 0.581742\n",
      "INFO:tensorflow:examples/sec: 18.6157\n",
      "INFO:tensorflow:global_step/sec: 0.581739\n",
      "INFO:tensorflow:examples/sec: 18.6157\n",
      "INFO:tensorflow:global_step/sec: 0.581228\n",
      "INFO:tensorflow:examples/sec: 18.5993\n",
      "INFO:tensorflow:Saving checkpoints for 35000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.54294\n",
      "INFO:tensorflow:examples/sec: 17.3741\n",
      "INFO:tensorflow:global_step/sec: 0.583564\n",
      "INFO:tensorflow:examples/sec: 18.6741\n",
      "INFO:tensorflow:global_step/sec: 0.582233\n",
      "INFO:tensorflow:examples/sec: 18.6315\n",
      "INFO:tensorflow:global_step/sec: 0.581517\n",
      "INFO:tensorflow:examples/sec: 18.6085\n",
      "INFO:tensorflow:global_step/sec: 0.582559\n",
      "INFO:tensorflow:examples/sec: 18.6419\n",
      "INFO:tensorflow:global_step/sec: 0.581831\n",
      "INFO:tensorflow:examples/sec: 18.6186\n",
      "INFO:tensorflow:global_step/sec: 0.581328\n",
      "INFO:tensorflow:examples/sec: 18.6025\n",
      "INFO:tensorflow:global_step/sec: 0.58132\n",
      "INFO:tensorflow:examples/sec: 18.6023\n",
      "INFO:tensorflow:global_step/sec: 0.580621\n",
      "INFO:tensorflow:examples/sec: 18.5799\n",
      "INFO:tensorflow:global_step/sec: 0.581864\n",
      "INFO:tensorflow:examples/sec: 18.6197\n",
      "INFO:tensorflow:Saving checkpoints for 36000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.543141\n",
      "INFO:tensorflow:examples/sec: 17.3805\n",
      "INFO:tensorflow:global_step/sec: 0.582709\n",
      "INFO:tensorflow:examples/sec: 18.6467\n",
      "INFO:tensorflow:global_step/sec: 0.582437\n",
      "INFO:tensorflow:examples/sec: 18.638\n",
      "INFO:tensorflow:global_step/sec: 0.582622\n",
      "INFO:tensorflow:examples/sec: 18.6439\n",
      "INFO:tensorflow:global_step/sec: 0.5822\n",
      "INFO:tensorflow:examples/sec: 18.6304\n",
      "INFO:tensorflow:global_step/sec: 0.58188\n",
      "INFO:tensorflow:examples/sec: 18.6202\n",
      "INFO:tensorflow:global_step/sec: 0.581389\n",
      "INFO:tensorflow:examples/sec: 18.6045\n",
      "INFO:tensorflow:global_step/sec: 0.581625\n",
      "INFO:tensorflow:examples/sec: 18.612\n",
      "INFO:tensorflow:global_step/sec: 0.581331\n",
      "INFO:tensorflow:examples/sec: 18.6026\n",
      "INFO:tensorflow:global_step/sec: 0.581357\n",
      "INFO:tensorflow:examples/sec: 18.6034\n",
      "INFO:tensorflow:Saving checkpoints for 37000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.543708\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:examples/sec: 17.3987\n",
      "INFO:tensorflow:global_step/sec: 0.583535\n",
      "INFO:tensorflow:examples/sec: 18.6731\n",
      "INFO:tensorflow:global_step/sec: 0.58215\n",
      "INFO:tensorflow:examples/sec: 18.6288\n",
      "INFO:tensorflow:global_step/sec: 0.581951\n",
      "INFO:tensorflow:examples/sec: 18.6224\n",
      "INFO:tensorflow:global_step/sec: 0.582295\n",
      "INFO:tensorflow:examples/sec: 18.6334\n",
      "INFO:tensorflow:global_step/sec: 0.581645\n",
      "INFO:tensorflow:examples/sec: 18.6126\n",
      "INFO:tensorflow:global_step/sec: 0.582195\n",
      "INFO:tensorflow:examples/sec: 18.6302\n",
      "INFO:tensorflow:global_step/sec: 0.581786\n",
      "INFO:tensorflow:examples/sec: 18.6171\n",
      "INFO:tensorflow:global_step/sec: 0.581987\n",
      "INFO:tensorflow:examples/sec: 18.6236\n",
      "INFO:tensorflow:global_step/sec: 0.581738\n",
      "INFO:tensorflow:examples/sec: 18.6156\n",
      "INFO:tensorflow:Saving checkpoints for 38000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.543909\n",
      "INFO:tensorflow:examples/sec: 17.4051\n",
      "INFO:tensorflow:global_step/sec: 0.58341\n",
      "INFO:tensorflow:examples/sec: 18.6691\n",
      "INFO:tensorflow:global_step/sec: 0.58207\n",
      "INFO:tensorflow:examples/sec: 18.6262\n",
      "INFO:tensorflow:global_step/sec: 0.581635\n",
      "INFO:tensorflow:examples/sec: 18.6123\n",
      "INFO:tensorflow:global_step/sec: 0.580903\n",
      "INFO:tensorflow:examples/sec: 18.5889\n",
      "INFO:tensorflow:global_step/sec: 0.581141\n",
      "INFO:tensorflow:examples/sec: 18.5965\n",
      "INFO:tensorflow:global_step/sec: 0.581821\n",
      "INFO:tensorflow:examples/sec: 18.6183\n",
      "INFO:tensorflow:global_step/sec: 0.581762\n",
      "INFO:tensorflow:examples/sec: 18.6164\n",
      "INFO:tensorflow:global_step/sec: 0.580963\n",
      "INFO:tensorflow:examples/sec: 18.5908\n",
      "INFO:tensorflow:global_step/sec: 0.581373\n",
      "INFO:tensorflow:examples/sec: 18.6039\n",
      "INFO:tensorflow:Saving checkpoints for 39000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.543081\n",
      "INFO:tensorflow:examples/sec: 17.3786\n",
      "INFO:tensorflow:global_step/sec: 0.582852\n",
      "INFO:tensorflow:examples/sec: 18.6513\n",
      "INFO:tensorflow:global_step/sec: 0.581428\n",
      "INFO:tensorflow:examples/sec: 18.6057\n",
      "INFO:tensorflow:global_step/sec: 0.581272\n",
      "INFO:tensorflow:examples/sec: 18.6007\n",
      "INFO:tensorflow:global_step/sec: 0.581599\n",
      "INFO:tensorflow:examples/sec: 18.6112\n",
      "INFO:tensorflow:global_step/sec: 0.581411\n",
      "INFO:tensorflow:examples/sec: 18.6052\n",
      "INFO:tensorflow:global_step/sec: 0.581092\n",
      "INFO:tensorflow:examples/sec: 18.5949\n",
      "INFO:tensorflow:global_step/sec: 0.580547\n",
      "INFO:tensorflow:examples/sec: 18.5775\n",
      "INFO:tensorflow:global_step/sec: 0.581127\n",
      "INFO:tensorflow:examples/sec: 18.5961\n",
      "INFO:tensorflow:global_step/sec: 0.581573\n",
      "INFO:tensorflow:examples/sec: 18.6103\n",
      "INFO:tensorflow:Saving checkpoints for 40000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.543371\n",
      "INFO:tensorflow:examples/sec: 17.3879\n",
      "INFO:tensorflow:global_step/sec: 0.583501\n",
      "INFO:tensorflow:examples/sec: 18.672\n",
      "INFO:tensorflow:global_step/sec: 0.582821\n",
      "INFO:tensorflow:examples/sec: 18.6503\n",
      "INFO:tensorflow:global_step/sec: 0.582131\n",
      "INFO:tensorflow:examples/sec: 18.6282\n",
      "INFO:tensorflow:global_step/sec: 0.581966\n",
      "INFO:tensorflow:examples/sec: 18.6229\n",
      "INFO:tensorflow:global_step/sec: 0.581108\n",
      "INFO:tensorflow:examples/sec: 18.5955\n",
      "INFO:tensorflow:global_step/sec: 0.58188\n",
      "INFO:tensorflow:examples/sec: 18.6202\n",
      "INFO:tensorflow:global_step/sec: 0.581758\n",
      "INFO:tensorflow:examples/sec: 18.6162\n",
      "INFO:tensorflow:global_step/sec: 0.582936\n",
      "INFO:tensorflow:examples/sec: 18.654\n",
      "INFO:tensorflow:global_step/sec: 0.581651\n",
      "INFO:tensorflow:examples/sec: 18.6128\n",
      "INFO:tensorflow:Saving checkpoints for 41000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.54411\n",
      "INFO:tensorflow:examples/sec: 17.4115\n",
      "INFO:tensorflow:global_step/sec: 0.5835\n",
      "INFO:tensorflow:examples/sec: 18.672\n",
      "INFO:tensorflow:global_step/sec: 0.581702\n",
      "INFO:tensorflow:examples/sec: 18.6145\n",
      "INFO:tensorflow:global_step/sec: 0.581642\n",
      "INFO:tensorflow:examples/sec: 18.6125\n",
      "INFO:tensorflow:global_step/sec: 0.581442\n",
      "INFO:tensorflow:examples/sec: 18.6061\n",
      "INFO:tensorflow:global_step/sec: 0.581538\n",
      "INFO:tensorflow:examples/sec: 18.6092\n",
      "INFO:tensorflow:global_step/sec: 0.581775\n",
      "INFO:tensorflow:examples/sec: 18.6168\n",
      "INFO:tensorflow:global_step/sec: 0.58222\n",
      "INFO:tensorflow:examples/sec: 18.631\n",
      "INFO:tensorflow:global_step/sec: 0.582679\n",
      "INFO:tensorflow:examples/sec: 18.6457\n",
      "INFO:tensorflow:global_step/sec: 0.581845\n",
      "INFO:tensorflow:examples/sec: 18.619\n",
      "INFO:tensorflow:Saving checkpoints for 42000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.544053\n",
      "INFO:tensorflow:examples/sec: 17.4097\n",
      "INFO:tensorflow:global_step/sec: 0.583994\n",
      "INFO:tensorflow:examples/sec: 18.6878\n",
      "INFO:tensorflow:global_step/sec: 0.582677\n",
      "INFO:tensorflow:examples/sec: 18.6457\n",
      "INFO:tensorflow:global_step/sec: 0.582418\n",
      "INFO:tensorflow:examples/sec: 18.6374\n",
      "INFO:tensorflow:global_step/sec: 0.581554\n",
      "INFO:tensorflow:examples/sec: 18.6097\n",
      "INFO:tensorflow:global_step/sec: 0.58161\n",
      "INFO:tensorflow:examples/sec: 18.6115\n",
      "INFO:tensorflow:global_step/sec: 0.582035\n",
      "INFO:tensorflow:examples/sec: 18.6251\n",
      "INFO:tensorflow:global_step/sec: 0.581325\n",
      "INFO:tensorflow:examples/sec: 18.6024\n",
      "INFO:tensorflow:global_step/sec: 0.581718\n",
      "INFO:tensorflow:examples/sec: 18.615\n",
      "INFO:tensorflow:global_step/sec: 0.581448\n",
      "INFO:tensorflow:examples/sec: 18.6063\n",
      "INFO:tensorflow:Saving checkpoints for 43000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.544121\n",
      "INFO:tensorflow:examples/sec: 17.4119\n",
      "INFO:tensorflow:global_step/sec: 0.583518\n",
      "INFO:tensorflow:examples/sec: 18.6726\n",
      "INFO:tensorflow:global_step/sec: 0.582339\n",
      "INFO:tensorflow:examples/sec: 18.6349\n",
      "INFO:tensorflow:global_step/sec: 0.582018\n",
      "INFO:tensorflow:examples/sec: 18.6246\n",
      "INFO:tensorflow:global_step/sec: 0.581453\n",
      "INFO:tensorflow:examples/sec: 18.6065\n",
      "INFO:tensorflow:global_step/sec: 0.581983\n",
      "INFO:tensorflow:examples/sec: 18.6234\n",
      "INFO:tensorflow:global_step/sec: 0.582079\n",
      "INFO:tensorflow:examples/sec: 18.6265\n",
      "INFO:tensorflow:global_step/sec: 0.581701\n",
      "INFO:tensorflow:examples/sec: 18.6144\n",
      "INFO:tensorflow:global_step/sec: 0.582061\n",
      "INFO:tensorflow:examples/sec: 18.6259\n",
      "INFO:tensorflow:global_step/sec: 0.582145\n",
      "INFO:tensorflow:examples/sec: 18.6286\n",
      "INFO:tensorflow:Saving checkpoints for 44000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.542646\n",
      "INFO:tensorflow:examples/sec: 17.3647\n",
      "INFO:tensorflow:global_step/sec: 0.583202\n",
      "INFO:tensorflow:examples/sec: 18.6625\n",
      "INFO:tensorflow:global_step/sec: 0.582491\n",
      "INFO:tensorflow:examples/sec: 18.6397\n",
      "INFO:tensorflow:global_step/sec: 0.582089\n",
      "INFO:tensorflow:examples/sec: 18.6268\n",
      "INFO:tensorflow:global_step/sec: 0.581555\n",
      "INFO:tensorflow:examples/sec: 18.6098\n",
      "INFO:tensorflow:global_step/sec: 0.581494\n",
      "INFO:tensorflow:examples/sec: 18.6078\n",
      "INFO:tensorflow:global_step/sec: 0.581524\n",
      "INFO:tensorflow:examples/sec: 18.6088\n",
      "INFO:tensorflow:global_step/sec: 0.581921\n",
      "INFO:tensorflow:examples/sec: 18.6215\n",
      "INFO:tensorflow:global_step/sec: 0.581074\n",
      "INFO:tensorflow:examples/sec: 18.5944\n",
      "INFO:tensorflow:global_step/sec: 0.580342\n",
      "INFO:tensorflow:examples/sec: 18.571\n",
      "INFO:tensorflow:Saving checkpoints for 45000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.54306\n",
      "INFO:tensorflow:examples/sec: 17.3779\n",
      "INFO:tensorflow:global_step/sec: 0.583757\n",
      "INFO:tensorflow:examples/sec: 18.6802\n",
      "INFO:tensorflow:global_step/sec: 0.581995\n",
      "INFO:tensorflow:examples/sec: 18.6238\n",
      "INFO:tensorflow:global_step/sec: 0.581386\n",
      "INFO:tensorflow:examples/sec: 18.6044\n",
      "INFO:tensorflow:global_step/sec: 0.58087\n",
      "INFO:tensorflow:examples/sec: 18.5878\n",
      "INFO:tensorflow:global_step/sec: 0.581123\n",
      "INFO:tensorflow:examples/sec: 18.5959\n",
      "INFO:tensorflow:global_step/sec: 0.58113\n",
      "INFO:tensorflow:examples/sec: 18.5962\n",
      "INFO:tensorflow:global_step/sec: 0.581382\n",
      "INFO:tensorflow:examples/sec: 18.6042\n",
      "INFO:tensorflow:global_step/sec: 0.581484\n",
      "INFO:tensorflow:examples/sec: 18.6075\n",
      "INFO:tensorflow:global_step/sec: 0.581894\n",
      "INFO:tensorflow:examples/sec: 18.6206\n",
      "INFO:tensorflow:Saving checkpoints for 46000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.542878\n",
      "INFO:tensorflow:examples/sec: 17.3721\n",
      "INFO:tensorflow:global_step/sec: 0.582057\n",
      "INFO:tensorflow:examples/sec: 18.6258\n",
      "INFO:tensorflow:global_step/sec: 0.58205\n",
      "INFO:tensorflow:examples/sec: 18.6256\n",
      "INFO:tensorflow:global_step/sec: 0.58153\n",
      "INFO:tensorflow:examples/sec: 18.609\n",
      "INFO:tensorflow:global_step/sec: 0.581288\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:examples/sec: 18.6012\n",
      "INFO:tensorflow:global_step/sec: 0.581128\n",
      "INFO:tensorflow:examples/sec: 18.5961\n",
      "INFO:tensorflow:global_step/sec: 0.580779\n",
      "INFO:tensorflow:examples/sec: 18.5849\n",
      "INFO:tensorflow:global_step/sec: 0.582397\n",
      "INFO:tensorflow:examples/sec: 18.6367\n",
      "INFO:tensorflow:global_step/sec: 0.580843\n",
      "INFO:tensorflow:examples/sec: 18.587\n",
      "INFO:tensorflow:global_step/sec: 0.581428\n",
      "INFO:tensorflow:examples/sec: 18.6057\n",
      "INFO:tensorflow:Saving checkpoints for 47000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.543528\n",
      "INFO:tensorflow:examples/sec: 17.3929\n",
      "INFO:tensorflow:global_step/sec: 0.582539\n",
      "INFO:tensorflow:examples/sec: 18.6412\n",
      "INFO:tensorflow:global_step/sec: 0.581609\n",
      "INFO:tensorflow:examples/sec: 18.6115\n",
      "INFO:tensorflow:global_step/sec: 0.581409\n",
      "INFO:tensorflow:examples/sec: 18.6051\n",
      "INFO:tensorflow:global_step/sec: 0.581821\n",
      "INFO:tensorflow:examples/sec: 18.6183\n",
      "INFO:tensorflow:global_step/sec: 0.581365\n",
      "INFO:tensorflow:examples/sec: 18.6037\n",
      "INFO:tensorflow:global_step/sec: 0.581495\n",
      "INFO:tensorflow:examples/sec: 18.6078\n",
      "INFO:tensorflow:global_step/sec: 0.581559\n",
      "INFO:tensorflow:examples/sec: 18.6099\n",
      "INFO:tensorflow:global_step/sec: 0.581342\n",
      "INFO:tensorflow:examples/sec: 18.6029\n",
      "INFO:tensorflow:global_step/sec: 0.580971\n",
      "INFO:tensorflow:examples/sec: 18.5911\n",
      "INFO:tensorflow:Saving checkpoints for 48000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.543649\n",
      "INFO:tensorflow:examples/sec: 17.3968\n",
      "INFO:tensorflow:global_step/sec: 0.581783\n",
      "INFO:tensorflow:examples/sec: 18.6171\n",
      "INFO:tensorflow:global_step/sec: 0.581056\n",
      "INFO:tensorflow:examples/sec: 18.5938\n",
      "INFO:tensorflow:global_step/sec: 0.580903\n",
      "INFO:tensorflow:examples/sec: 18.5889\n",
      "INFO:tensorflow:global_step/sec: 0.581444\n",
      "INFO:tensorflow:examples/sec: 18.6062\n",
      "INFO:tensorflow:global_step/sec: 0.581597\n",
      "INFO:tensorflow:examples/sec: 18.6111\n",
      "INFO:tensorflow:global_step/sec: 0.581195\n",
      "INFO:tensorflow:examples/sec: 18.5982\n",
      "INFO:tensorflow:global_step/sec: 0.580757\n",
      "INFO:tensorflow:examples/sec: 18.5842\n",
      "INFO:tensorflow:global_step/sec: 0.580773\n",
      "INFO:tensorflow:examples/sec: 18.5847\n",
      "INFO:tensorflow:global_step/sec: 0.581242\n",
      "INFO:tensorflow:examples/sec: 18.5998\n",
      "INFO:tensorflow:Saving checkpoints for 49000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.543363\n",
      "INFO:tensorflow:examples/sec: 17.3876\n",
      "INFO:tensorflow:global_step/sec: 0.581846\n",
      "INFO:tensorflow:examples/sec: 18.6191\n",
      "INFO:tensorflow:global_step/sec: 0.580905\n",
      "INFO:tensorflow:examples/sec: 18.589\n",
      "INFO:tensorflow:global_step/sec: 0.579661\n",
      "INFO:tensorflow:examples/sec: 18.5491\n",
      "INFO:tensorflow:global_step/sec: 0.58147\n",
      "INFO:tensorflow:examples/sec: 18.607\n",
      "INFO:tensorflow:global_step/sec: 0.580683\n",
      "INFO:tensorflow:examples/sec: 18.5819\n",
      "INFO:tensorflow:global_step/sec: 0.581049\n",
      "INFO:tensorflow:examples/sec: 18.5936\n",
      "INFO:tensorflow:global_step/sec: 0.580492\n",
      "INFO:tensorflow:examples/sec: 18.5757\n",
      "INFO:tensorflow:global_step/sec: 0.580943\n",
      "INFO:tensorflow:examples/sec: 18.5902\n",
      "INFO:tensorflow:global_step/sec: 0.581203\n",
      "INFO:tensorflow:examples/sec: 18.5985\n",
      "INFO:tensorflow:Saving checkpoints for 50000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.543161\n",
      "INFO:tensorflow:examples/sec: 17.3811\n",
      "INFO:tensorflow:global_step/sec: 0.582438\n",
      "INFO:tensorflow:examples/sec: 18.638\n",
      "INFO:tensorflow:global_step/sec: 0.581232\n",
      "INFO:tensorflow:examples/sec: 18.5994\n",
      "INFO:tensorflow:global_step/sec: 0.580508\n",
      "INFO:tensorflow:examples/sec: 18.5763\n",
      "INFO:tensorflow:global_step/sec: 0.581947\n",
      "INFO:tensorflow:examples/sec: 18.6223\n",
      "INFO:tensorflow:global_step/sec: 0.581293\n",
      "INFO:tensorflow:examples/sec: 18.6014\n",
      "INFO:tensorflow:global_step/sec: 0.581058\n",
      "INFO:tensorflow:examples/sec: 18.5939\n",
      "INFO:tensorflow:global_step/sec: 0.581514\n",
      "INFO:tensorflow:examples/sec: 18.6084\n",
      "INFO:tensorflow:global_step/sec: 0.581058\n",
      "INFO:tensorflow:examples/sec: 18.5939\n",
      "INFO:tensorflow:global_step/sec: 0.581821\n",
      "INFO:tensorflow:examples/sec: 18.6183\n",
      "INFO:tensorflow:Saving checkpoints for 51000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.542647\n",
      "INFO:tensorflow:examples/sec: 17.3647\n",
      "INFO:tensorflow:global_step/sec: 0.583648\n",
      "INFO:tensorflow:examples/sec: 18.6767\n",
      "INFO:tensorflow:global_step/sec: 0.582231\n",
      "INFO:tensorflow:examples/sec: 18.6314\n",
      "INFO:tensorflow:global_step/sec: 0.581738\n",
      "INFO:tensorflow:examples/sec: 18.6156\n",
      "INFO:tensorflow:global_step/sec: 0.582385\n",
      "INFO:tensorflow:examples/sec: 18.6363\n",
      "INFO:tensorflow:global_step/sec: 0.581756\n",
      "INFO:tensorflow:examples/sec: 18.6162\n",
      "INFO:tensorflow:global_step/sec: 0.581739\n",
      "INFO:tensorflow:examples/sec: 18.6156\n",
      "INFO:tensorflow:global_step/sec: 0.581597\n",
      "INFO:tensorflow:examples/sec: 18.6111\n",
      "INFO:tensorflow:global_step/sec: 0.580811\n",
      "INFO:tensorflow:examples/sec: 18.5859\n",
      "INFO:tensorflow:global_step/sec: 0.581266\n",
      "INFO:tensorflow:examples/sec: 18.6005\n",
      "INFO:tensorflow:Saving checkpoints for 52000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.542932\n",
      "INFO:tensorflow:examples/sec: 17.3738\n",
      "INFO:tensorflow:global_step/sec: 0.583128\n",
      "INFO:tensorflow:examples/sec: 18.6601\n",
      "INFO:tensorflow:global_step/sec: 0.581275\n",
      "INFO:tensorflow:examples/sec: 18.6008\n",
      "INFO:tensorflow:global_step/sec: 0.581612\n",
      "INFO:tensorflow:examples/sec: 18.6116\n",
      "INFO:tensorflow:global_step/sec: 0.580743\n",
      "INFO:tensorflow:examples/sec: 18.5838\n",
      "INFO:tensorflow:global_step/sec: 0.580859\n",
      "INFO:tensorflow:examples/sec: 18.5875\n",
      "INFO:tensorflow:global_step/sec: 0.580397\n",
      "INFO:tensorflow:examples/sec: 18.5727\n",
      "INFO:tensorflow:global_step/sec: 0.580748\n",
      "INFO:tensorflow:examples/sec: 18.5839\n",
      "INFO:tensorflow:global_step/sec: 0.58148\n",
      "INFO:tensorflow:examples/sec: 18.6074\n",
      "INFO:tensorflow:global_step/sec: 0.580209\n",
      "INFO:tensorflow:examples/sec: 18.5667\n",
      "INFO:tensorflow:Saving checkpoints for 53000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.543306\n",
      "INFO:tensorflow:examples/sec: 17.3858\n",
      "INFO:tensorflow:global_step/sec: 0.583114\n",
      "INFO:tensorflow:examples/sec: 18.6597\n",
      "INFO:tensorflow:global_step/sec: 0.581324\n",
      "INFO:tensorflow:examples/sec: 18.6024\n",
      "INFO:tensorflow:global_step/sec: 0.58174\n",
      "INFO:tensorflow:examples/sec: 18.6157\n",
      "INFO:tensorflow:global_step/sec: 0.582492\n",
      "INFO:tensorflow:examples/sec: 18.6397\n",
      "INFO:tensorflow:global_step/sec: 0.581395\n",
      "INFO:tensorflow:examples/sec: 18.6046\n",
      "INFO:tensorflow:global_step/sec: 0.580102\n",
      "INFO:tensorflow:examples/sec: 18.5633\n",
      "INFO:tensorflow:global_step/sec: 0.5815\n",
      "INFO:tensorflow:examples/sec: 18.608\n",
      "INFO:tensorflow:global_step/sec: 0.581249\n",
      "INFO:tensorflow:examples/sec: 18.6\n",
      "INFO:tensorflow:global_step/sec: 0.581551\n",
      "INFO:tensorflow:examples/sec: 18.6096\n",
      "INFO:tensorflow:Saving checkpoints for 54000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.542852\n",
      "INFO:tensorflow:examples/sec: 17.3713\n",
      "INFO:tensorflow:global_step/sec: 0.582913\n",
      "INFO:tensorflow:examples/sec: 18.6532\n",
      "INFO:tensorflow:global_step/sec: 0.581629\n",
      "INFO:tensorflow:examples/sec: 18.6121\n",
      "INFO:tensorflow:global_step/sec: 0.582368\n",
      "INFO:tensorflow:examples/sec: 18.6358\n",
      "INFO:tensorflow:global_step/sec: 0.581585\n",
      "INFO:tensorflow:examples/sec: 18.6107\n",
      "INFO:tensorflow:global_step/sec: 0.581642\n",
      "INFO:tensorflow:examples/sec: 18.6126\n",
      "INFO:tensorflow:global_step/sec: 0.581076\n",
      "INFO:tensorflow:examples/sec: 18.5944\n",
      "INFO:tensorflow:global_step/sec: 0.58064\n",
      "INFO:tensorflow:examples/sec: 18.5805\n",
      "INFO:tensorflow:global_step/sec: 0.581002\n",
      "INFO:tensorflow:examples/sec: 18.5921\n",
      "INFO:tensorflow:global_step/sec: 0.581182\n",
      "INFO:tensorflow:examples/sec: 18.5978\n",
      "INFO:tensorflow:Saving checkpoints for 55000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.543297\n",
      "INFO:tensorflow:examples/sec: 17.3855\n",
      "INFO:tensorflow:global_step/sec: 0.583122\n",
      "INFO:tensorflow:examples/sec: 18.6599\n",
      "INFO:tensorflow:global_step/sec: 0.581831\n",
      "INFO:tensorflow:examples/sec: 18.6186\n",
      "INFO:tensorflow:global_step/sec: 0.580793\n",
      "INFO:tensorflow:examples/sec: 18.5854\n",
      "INFO:tensorflow:global_step/sec: 0.581099\n",
      "INFO:tensorflow:examples/sec: 18.5952\n",
      "INFO:tensorflow:global_step/sec: 0.581157\n",
      "INFO:tensorflow:examples/sec: 18.597\n",
      "INFO:tensorflow:global_step/sec: 0.580998\n",
      "INFO:tensorflow:examples/sec: 18.5919\n",
      "INFO:tensorflow:global_step/sec: 0.581143\n",
      "INFO:tensorflow:examples/sec: 18.5966\n",
      "INFO:tensorflow:global_step/sec: 0.581101\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:examples/sec: 18.5952\n",
      "INFO:tensorflow:global_step/sec: 0.58153\n",
      "INFO:tensorflow:examples/sec: 18.609\n",
      "INFO:tensorflow:Saving checkpoints for 56000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.543483\n",
      "INFO:tensorflow:examples/sec: 17.3915\n",
      "INFO:tensorflow:global_step/sec: 0.58375\n",
      "INFO:tensorflow:examples/sec: 18.68\n",
      "INFO:tensorflow:global_step/sec: 0.582632\n",
      "INFO:tensorflow:examples/sec: 18.6442\n",
      "INFO:tensorflow:global_step/sec: 0.581487\n",
      "INFO:tensorflow:examples/sec: 18.6076\n",
      "INFO:tensorflow:global_step/sec: 0.582093\n",
      "INFO:tensorflow:examples/sec: 18.627\n",
      "INFO:tensorflow:global_step/sec: 0.581732\n",
      "INFO:tensorflow:examples/sec: 18.6154\n",
      "INFO:tensorflow:global_step/sec: 0.581866\n",
      "INFO:tensorflow:examples/sec: 18.6197\n",
      "INFO:tensorflow:global_step/sec: 0.581304\n",
      "INFO:tensorflow:examples/sec: 18.6017\n",
      "INFO:tensorflow:global_step/sec: 0.580056\n",
      "INFO:tensorflow:examples/sec: 18.5618\n",
      "INFO:tensorflow:global_step/sec: 0.58153\n",
      "INFO:tensorflow:examples/sec: 18.609\n",
      "INFO:tensorflow:Saving checkpoints for 57000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.542779\n",
      "INFO:tensorflow:examples/sec: 17.3689\n",
      "INFO:tensorflow:global_step/sec: 0.583874\n",
      "INFO:tensorflow:examples/sec: 18.684\n",
      "INFO:tensorflow:global_step/sec: 0.581895\n",
      "INFO:tensorflow:examples/sec: 18.6206\n",
      "INFO:tensorflow:global_step/sec: 0.580213\n",
      "INFO:tensorflow:examples/sec: 18.5668\n",
      "INFO:tensorflow:global_step/sec: 0.581529\n",
      "INFO:tensorflow:examples/sec: 18.6089\n",
      "INFO:tensorflow:global_step/sec: 0.581847\n",
      "INFO:tensorflow:examples/sec: 18.6191\n",
      "INFO:tensorflow:global_step/sec: 0.580743\n",
      "INFO:tensorflow:examples/sec: 18.5838\n",
      "INFO:tensorflow:global_step/sec: 0.581244\n",
      "INFO:tensorflow:examples/sec: 18.5998\n",
      "INFO:tensorflow:global_step/sec: 0.581567\n",
      "INFO:tensorflow:examples/sec: 18.6102\n",
      "INFO:tensorflow:global_step/sec: 0.581829\n",
      "INFO:tensorflow:examples/sec: 18.6185\n",
      "INFO:tensorflow:Saving checkpoints for 58000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.54331\n",
      "INFO:tensorflow:examples/sec: 17.3859\n",
      "INFO:tensorflow:global_step/sec: 0.58369\n",
      "INFO:tensorflow:examples/sec: 18.6781\n",
      "INFO:tensorflow:global_step/sec: 0.58258\n",
      "INFO:tensorflow:examples/sec: 18.6426\n",
      "INFO:tensorflow:global_step/sec: 0.581967\n",
      "INFO:tensorflow:examples/sec: 18.623\n",
      "INFO:tensorflow:global_step/sec: 0.582187\n",
      "INFO:tensorflow:examples/sec: 18.63\n",
      "INFO:tensorflow:global_step/sec: 0.582302\n",
      "INFO:tensorflow:examples/sec: 18.6337\n",
      "INFO:tensorflow:global_step/sec: 0.581502\n",
      "INFO:tensorflow:examples/sec: 18.6081\n",
      "INFO:tensorflow:global_step/sec: 0.582081\n",
      "INFO:tensorflow:examples/sec: 18.6266\n",
      "INFO:tensorflow:global_step/sec: 0.582006\n",
      "INFO:tensorflow:examples/sec: 18.6242\n",
      "INFO:tensorflow:global_step/sec: 0.581837\n",
      "INFO:tensorflow:examples/sec: 18.6188\n",
      "INFO:tensorflow:Saving checkpoints for 59000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.544546\n",
      "INFO:tensorflow:examples/sec: 17.4255\n",
      "INFO:tensorflow:global_step/sec: 0.583804\n",
      "INFO:tensorflow:examples/sec: 18.6817\n",
      "INFO:tensorflow:global_step/sec: 0.582644\n",
      "INFO:tensorflow:examples/sec: 18.6446\n",
      "INFO:tensorflow:global_step/sec: 0.582007\n",
      "INFO:tensorflow:examples/sec: 18.6242\n",
      "INFO:tensorflow:global_step/sec: 0.582362\n",
      "INFO:tensorflow:examples/sec: 18.6356\n",
      "INFO:tensorflow:global_step/sec: 0.582365\n",
      "INFO:tensorflow:examples/sec: 18.6357\n",
      "INFO:tensorflow:global_step/sec: 0.581582\n",
      "INFO:tensorflow:examples/sec: 18.6106\n",
      "INFO:tensorflow:global_step/sec: 0.582262\n",
      "INFO:tensorflow:examples/sec: 18.6324\n",
      "INFO:tensorflow:global_step/sec: 0.581236\n",
      "INFO:tensorflow:examples/sec: 18.5996\n",
      "INFO:tensorflow:global_step/sec: 0.581363\n",
      "INFO:tensorflow:examples/sec: 18.6036\n",
      "INFO:tensorflow:Saving checkpoints for 60000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.531434\n",
      "INFO:tensorflow:examples/sec: 17.0059\n",
      "INFO:tensorflow:global_step/sec: 0.584229\n",
      "INFO:tensorflow:examples/sec: 18.6953\n",
      "INFO:tensorflow:global_step/sec: 0.58177\n",
      "INFO:tensorflow:examples/sec: 18.6166\n",
      "INFO:tensorflow:global_step/sec: 0.58104\n",
      "INFO:tensorflow:examples/sec: 18.5933\n",
      "INFO:tensorflow:global_step/sec: 0.581689\n",
      "INFO:tensorflow:examples/sec: 18.614\n",
      "INFO:tensorflow:global_step/sec: 0.580946\n",
      "INFO:tensorflow:examples/sec: 18.5903\n",
      "INFO:tensorflow:global_step/sec: 0.581178\n",
      "INFO:tensorflow:examples/sec: 18.5977\n",
      "INFO:tensorflow:global_step/sec: 0.58135\n",
      "INFO:tensorflow:examples/sec: 18.6032\n",
      "INFO:tensorflow:global_step/sec: 0.58127\n",
      "INFO:tensorflow:examples/sec: 18.6006\n",
      "INFO:tensorflow:global_step/sec: 0.580727\n",
      "INFO:tensorflow:examples/sec: 18.5833\n",
      "INFO:tensorflow:Saving checkpoints for 61000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.543146\n",
      "INFO:tensorflow:examples/sec: 17.3807\n",
      "INFO:tensorflow:global_step/sec: 0.581677\n",
      "INFO:tensorflow:examples/sec: 18.6137\n",
      "INFO:tensorflow:global_step/sec: 0.581545\n",
      "INFO:tensorflow:examples/sec: 18.6094\n",
      "INFO:tensorflow:global_step/sec: 0.582041\n",
      "INFO:tensorflow:examples/sec: 18.6253\n",
      "INFO:tensorflow:global_step/sec: 0.582141\n",
      "INFO:tensorflow:examples/sec: 18.6285\n",
      "INFO:tensorflow:global_step/sec: 0.581245\n",
      "INFO:tensorflow:examples/sec: 18.5999\n",
      "INFO:tensorflow:global_step/sec: 0.582108\n",
      "INFO:tensorflow:examples/sec: 18.6275\n",
      "INFO:tensorflow:global_step/sec: 0.581597\n",
      "INFO:tensorflow:examples/sec: 18.6111\n",
      "INFO:tensorflow:global_step/sec: 0.581535\n",
      "INFO:tensorflow:examples/sec: 18.6091\n",
      "INFO:tensorflow:global_step/sec: 0.581913\n",
      "INFO:tensorflow:examples/sec: 18.6212\n",
      "INFO:tensorflow:Saving checkpoints for 62000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.544293\n",
      "INFO:tensorflow:examples/sec: 17.4174\n",
      "INFO:tensorflow:global_step/sec: 0.583996\n",
      "INFO:tensorflow:examples/sec: 18.6879\n",
      "INFO:tensorflow:global_step/sec: 0.582277\n",
      "INFO:tensorflow:examples/sec: 18.6329\n",
      "INFO:tensorflow:global_step/sec: 0.581872\n",
      "INFO:tensorflow:examples/sec: 18.6199\n",
      "INFO:tensorflow:global_step/sec: 0.581544\n",
      "INFO:tensorflow:examples/sec: 18.6094\n",
      "INFO:tensorflow:global_step/sec: 0.581686\n",
      "INFO:tensorflow:examples/sec: 18.6139\n",
      "INFO:tensorflow:global_step/sec: 0.581479\n",
      "INFO:tensorflow:examples/sec: 18.6073\n",
      "INFO:tensorflow:global_step/sec: 0.581493\n",
      "INFO:tensorflow:examples/sec: 18.6078\n",
      "INFO:tensorflow:global_step/sec: 0.582489\n",
      "INFO:tensorflow:examples/sec: 18.6396\n",
      "INFO:tensorflow:global_step/sec: 0.581679\n",
      "INFO:tensorflow:examples/sec: 18.6137\n",
      "INFO:tensorflow:Saving checkpoints for 63000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.544137\n",
      "INFO:tensorflow:examples/sec: 17.4124\n",
      "INFO:tensorflow:global_step/sec: 0.582804\n",
      "INFO:tensorflow:examples/sec: 18.6497\n",
      "INFO:tensorflow:global_step/sec: 0.582481\n",
      "INFO:tensorflow:examples/sec: 18.6394\n",
      "INFO:tensorflow:global_step/sec: 0.581832\n",
      "INFO:tensorflow:examples/sec: 18.6186\n",
      "INFO:tensorflow:global_step/sec: 0.582226\n",
      "INFO:tensorflow:examples/sec: 18.6312\n",
      "INFO:tensorflow:global_step/sec: 0.582167\n",
      "INFO:tensorflow:examples/sec: 18.6294\n",
      "INFO:tensorflow:global_step/sec: 0.580439\n",
      "INFO:tensorflow:examples/sec: 18.574\n",
      "INFO:tensorflow:global_step/sec: 0.581927\n",
      "INFO:tensorflow:examples/sec: 18.6217\n",
      "INFO:tensorflow:global_step/sec: 0.581137\n",
      "INFO:tensorflow:examples/sec: 18.5964\n",
      "INFO:tensorflow:global_step/sec: 0.5822\n",
      "INFO:tensorflow:examples/sec: 18.6304\n",
      "INFO:tensorflow:Saving checkpoints for 64000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.544154\n",
      "INFO:tensorflow:examples/sec: 17.4129\n",
      "INFO:tensorflow:global_step/sec: 0.583543\n",
      "INFO:tensorflow:examples/sec: 18.6734\n",
      "INFO:tensorflow:global_step/sec: 0.581382\n",
      "INFO:tensorflow:examples/sec: 18.6042\n",
      "INFO:tensorflow:global_step/sec: 0.580934\n",
      "INFO:tensorflow:examples/sec: 18.5899\n",
      "INFO:tensorflow:global_step/sec: 0.582664\n",
      "INFO:tensorflow:examples/sec: 18.6452\n",
      "INFO:tensorflow:global_step/sec: 0.581539\n",
      "INFO:tensorflow:examples/sec: 18.6092\n",
      "INFO:tensorflow:global_step/sec: 0.581502\n",
      "INFO:tensorflow:examples/sec: 18.6081\n",
      "INFO:tensorflow:global_step/sec: 0.581339\n",
      "INFO:tensorflow:examples/sec: 18.6028\n",
      "INFO:tensorflow:global_step/sec: 0.582428\n",
      "INFO:tensorflow:examples/sec: 18.6377\n",
      "INFO:tensorflow:global_step/sec: 0.582116\n",
      "INFO:tensorflow:examples/sec: 18.6277\n",
      "INFO:tensorflow:Saving checkpoints for 65000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.542758\n",
      "INFO:tensorflow:examples/sec: 17.3682\n",
      "INFO:tensorflow:global_step/sec: 0.583262\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:examples/sec: 18.6644\n",
      "INFO:tensorflow:global_step/sec: 0.581815\n",
      "INFO:tensorflow:examples/sec: 18.6181\n",
      "INFO:tensorflow:global_step/sec: 0.581285\n",
      "INFO:tensorflow:examples/sec: 18.6011\n",
      "INFO:tensorflow:global_step/sec: 0.581335\n",
      "INFO:tensorflow:examples/sec: 18.6027\n",
      "INFO:tensorflow:global_step/sec: 0.5818\n",
      "INFO:tensorflow:examples/sec: 18.6176\n",
      "INFO:tensorflow:global_step/sec: 0.58208\n",
      "INFO:tensorflow:examples/sec: 18.6266\n",
      "INFO:tensorflow:global_step/sec: 0.580954\n",
      "INFO:tensorflow:examples/sec: 18.5905\n",
      "INFO:tensorflow:global_step/sec: 0.581261\n",
      "INFO:tensorflow:examples/sec: 18.6004\n",
      "INFO:tensorflow:global_step/sec: 0.581416\n",
      "INFO:tensorflow:examples/sec: 18.6053\n",
      "INFO:tensorflow:Saving checkpoints for 66000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.542988\n",
      "INFO:tensorflow:examples/sec: 17.3756\n",
      "INFO:tensorflow:global_step/sec: 0.583169\n",
      "INFO:tensorflow:examples/sec: 18.6614\n",
      "INFO:tensorflow:global_step/sec: 0.582376\n",
      "INFO:tensorflow:examples/sec: 18.636\n",
      "INFO:tensorflow:global_step/sec: 0.581989\n",
      "INFO:tensorflow:examples/sec: 18.6236\n",
      "INFO:tensorflow:global_step/sec: 0.581463\n",
      "INFO:tensorflow:examples/sec: 18.6068\n",
      "INFO:tensorflow:global_step/sec: 0.581547\n",
      "INFO:tensorflow:examples/sec: 18.6095\n",
      "INFO:tensorflow:global_step/sec: 0.582005\n",
      "INFO:tensorflow:examples/sec: 18.6241\n",
      "INFO:tensorflow:global_step/sec: 0.582274\n",
      "INFO:tensorflow:examples/sec: 18.6328\n",
      "INFO:tensorflow:global_step/sec: 0.581292\n",
      "INFO:tensorflow:examples/sec: 18.6013\n",
      "INFO:tensorflow:global_step/sec: 0.582032\n",
      "INFO:tensorflow:examples/sec: 18.625\n",
      "INFO:tensorflow:Saving checkpoints for 67000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.543435\n",
      "INFO:tensorflow:examples/sec: 17.3899\n",
      "INFO:tensorflow:global_step/sec: 0.583501\n",
      "INFO:tensorflow:examples/sec: 18.672\n",
      "INFO:tensorflow:global_step/sec: 0.583018\n",
      "INFO:tensorflow:examples/sec: 18.6566\n",
      "INFO:tensorflow:global_step/sec: 0.582521\n",
      "INFO:tensorflow:examples/sec: 18.6407\n",
      "INFO:tensorflow:global_step/sec: 0.581882\n",
      "INFO:tensorflow:examples/sec: 18.6202\n",
      "INFO:tensorflow:global_step/sec: 0.58159\n",
      "INFO:tensorflow:examples/sec: 18.6109\n",
      "INFO:tensorflow:global_step/sec: 0.581596\n",
      "INFO:tensorflow:examples/sec: 18.6111\n",
      "INFO:tensorflow:global_step/sec: 0.582099\n",
      "INFO:tensorflow:examples/sec: 18.6272\n",
      "INFO:tensorflow:global_step/sec: 0.581686\n",
      "INFO:tensorflow:examples/sec: 18.614\n",
      "INFO:tensorflow:global_step/sec: 0.581256\n",
      "INFO:tensorflow:examples/sec: 18.6002\n",
      "INFO:tensorflow:Saving checkpoints for 68000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.543938\n",
      "INFO:tensorflow:examples/sec: 17.406\n",
      "INFO:tensorflow:global_step/sec: 0.583869\n",
      "INFO:tensorflow:examples/sec: 18.6838\n",
      "INFO:tensorflow:global_step/sec: 0.581302\n",
      "INFO:tensorflow:examples/sec: 18.6017\n",
      "INFO:tensorflow:global_step/sec: 0.581657\n",
      "INFO:tensorflow:examples/sec: 18.613\n",
      "INFO:tensorflow:global_step/sec: 0.581558\n",
      "INFO:tensorflow:examples/sec: 18.6099\n",
      "INFO:tensorflow:global_step/sec: 0.582573\n",
      "INFO:tensorflow:examples/sec: 18.6423\n",
      "INFO:tensorflow:global_step/sec: 0.582258\n",
      "INFO:tensorflow:examples/sec: 18.6323\n",
      "INFO:tensorflow:global_step/sec: 0.58194\n",
      "INFO:tensorflow:examples/sec: 18.6221\n",
      "INFO:tensorflow:global_step/sec: 0.581781\n",
      "INFO:tensorflow:examples/sec: 18.617\n",
      "INFO:tensorflow:global_step/sec: 0.581831\n",
      "INFO:tensorflow:examples/sec: 18.6186\n",
      "INFO:tensorflow:Saving checkpoints for 69000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.544004\n",
      "INFO:tensorflow:examples/sec: 17.4081\n",
      "INFO:tensorflow:global_step/sec: 0.583346\n",
      "INFO:tensorflow:examples/sec: 18.6671\n",
      "INFO:tensorflow:global_step/sec: 0.582262\n",
      "INFO:tensorflow:examples/sec: 18.6324\n",
      "INFO:tensorflow:global_step/sec: 0.581825\n",
      "INFO:tensorflow:examples/sec: 18.6184\n",
      "INFO:tensorflow:global_step/sec: 0.582179\n",
      "INFO:tensorflow:examples/sec: 18.6297\n",
      "INFO:tensorflow:global_step/sec: 0.580572\n",
      "INFO:tensorflow:examples/sec: 18.5783\n",
      "INFO:tensorflow:global_step/sec: 0.581051\n",
      "INFO:tensorflow:examples/sec: 18.5936\n",
      "INFO:tensorflow:global_step/sec: 0.581138\n",
      "INFO:tensorflow:examples/sec: 18.5964\n",
      "INFO:tensorflow:global_step/sec: 0.582119\n",
      "INFO:tensorflow:examples/sec: 18.6278\n",
      "INFO:tensorflow:global_step/sec: 0.581196\n",
      "INFO:tensorflow:examples/sec: 18.5983\n",
      "INFO:tensorflow:Saving checkpoints for 70000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.543528\n",
      "INFO:tensorflow:examples/sec: 17.3929\n",
      "INFO:tensorflow:global_step/sec: 0.582271\n",
      "INFO:tensorflow:examples/sec: 18.6327\n",
      "INFO:tensorflow:global_step/sec: 0.582633\n",
      "INFO:tensorflow:examples/sec: 18.6443\n",
      "INFO:tensorflow:global_step/sec: 0.582036\n",
      "INFO:tensorflow:examples/sec: 18.6252\n",
      "INFO:tensorflow:global_step/sec: 0.581633\n",
      "INFO:tensorflow:examples/sec: 18.6123\n",
      "INFO:tensorflow:global_step/sec: 0.581423\n",
      "INFO:tensorflow:examples/sec: 18.6055\n",
      "INFO:tensorflow:global_step/sec: 0.582427\n",
      "INFO:tensorflow:examples/sec: 18.6377\n",
      "INFO:tensorflow:global_step/sec: 0.582007\n",
      "INFO:tensorflow:examples/sec: 18.6242\n",
      "INFO:tensorflow:global_step/sec: 0.582424\n",
      "INFO:tensorflow:examples/sec: 18.6376\n",
      "INFO:tensorflow:global_step/sec: 0.581638\n",
      "INFO:tensorflow:examples/sec: 18.6124\n",
      "INFO:tensorflow:Saving checkpoints for 71000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.543248\n",
      "INFO:tensorflow:examples/sec: 17.3839\n",
      "INFO:tensorflow:global_step/sec: 0.5839\n",
      "INFO:tensorflow:examples/sec: 18.6848\n",
      "INFO:tensorflow:global_step/sec: 0.582096\n",
      "INFO:tensorflow:examples/sec: 18.6271\n",
      "INFO:tensorflow:global_step/sec: 0.58155\n",
      "INFO:tensorflow:examples/sec: 18.6096\n",
      "INFO:tensorflow:global_step/sec: 0.581069\n",
      "INFO:tensorflow:examples/sec: 18.5942\n",
      "INFO:tensorflow:global_step/sec: 0.581831\n",
      "INFO:tensorflow:examples/sec: 18.6186\n",
      "INFO:tensorflow:global_step/sec: 0.58238\n",
      "INFO:tensorflow:examples/sec: 18.6362\n",
      "INFO:tensorflow:global_step/sec: 0.582035\n",
      "INFO:tensorflow:examples/sec: 18.6251\n",
      "INFO:tensorflow:global_step/sec: 0.582124\n",
      "INFO:tensorflow:examples/sec: 18.628\n",
      "INFO:tensorflow:global_step/sec: 0.581539\n",
      "INFO:tensorflow:examples/sec: 18.6093\n",
      "INFO:tensorflow:Saving checkpoints for 72000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.544322\n",
      "INFO:tensorflow:examples/sec: 17.4183\n",
      "INFO:tensorflow:global_step/sec: 0.583183\n",
      "INFO:tensorflow:examples/sec: 18.6619\n",
      "INFO:tensorflow:global_step/sec: 0.581771\n",
      "INFO:tensorflow:examples/sec: 18.6167\n",
      "INFO:tensorflow:global_step/sec: 0.582898\n",
      "INFO:tensorflow:examples/sec: 18.6527\n",
      "INFO:tensorflow:global_step/sec: 0.581245\n",
      "INFO:tensorflow:examples/sec: 18.5998\n",
      "INFO:tensorflow:global_step/sec: 0.580955\n",
      "INFO:tensorflow:examples/sec: 18.5906\n",
      "INFO:tensorflow:global_step/sec: 0.581405\n",
      "INFO:tensorflow:examples/sec: 18.605\n",
      "INFO:tensorflow:global_step/sec: 0.581242\n",
      "INFO:tensorflow:examples/sec: 18.5997\n",
      "INFO:tensorflow:global_step/sec: 0.581123\n",
      "INFO:tensorflow:examples/sec: 18.5959\n",
      "INFO:tensorflow:global_step/sec: 0.581598\n",
      "INFO:tensorflow:examples/sec: 18.6111\n",
      "INFO:tensorflow:Saving checkpoints for 73000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.543271\n",
      "INFO:tensorflow:examples/sec: 17.3847\n",
      "INFO:tensorflow:global_step/sec: 0.582272\n",
      "INFO:tensorflow:examples/sec: 18.6327\n",
      "INFO:tensorflow:global_step/sec: 0.582307\n",
      "INFO:tensorflow:examples/sec: 18.6338\n",
      "INFO:tensorflow:global_step/sec: 0.581587\n",
      "INFO:tensorflow:examples/sec: 18.6108\n",
      "INFO:tensorflow:global_step/sec: 0.581186\n",
      "INFO:tensorflow:examples/sec: 18.598\n",
      "INFO:tensorflow:global_step/sec: 0.581125\n",
      "INFO:tensorflow:examples/sec: 18.596\n",
      "INFO:tensorflow:global_step/sec: 0.581344\n",
      "INFO:tensorflow:examples/sec: 18.603\n",
      "INFO:tensorflow:global_step/sec: 0.580643\n",
      "INFO:tensorflow:examples/sec: 18.5806\n",
      "INFO:tensorflow:global_step/sec: 0.581401\n",
      "INFO:tensorflow:examples/sec: 18.6048\n",
      "INFO:tensorflow:global_step/sec: 0.581297\n",
      "INFO:tensorflow:examples/sec: 18.6015\n",
      "INFO:tensorflow:Saving checkpoints for 74000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.542783\n",
      "INFO:tensorflow:examples/sec: 17.3691\n",
      "INFO:tensorflow:global_step/sec: 0.583897\n",
      "INFO:tensorflow:examples/sec: 18.6847\n",
      "INFO:tensorflow:global_step/sec: 0.582979\n",
      "INFO:tensorflow:examples/sec: 18.6553\n",
      "INFO:tensorflow:global_step/sec: 0.582045\n",
      "INFO:tensorflow:examples/sec: 18.6255\n",
      "INFO:tensorflow:global_step/sec: 0.583066\n",
      "INFO:tensorflow:examples/sec: 18.6581\n",
      "INFO:tensorflow:global_step/sec: 0.582238\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:examples/sec: 18.6316\n",
      "INFO:tensorflow:global_step/sec: 0.581981\n",
      "INFO:tensorflow:examples/sec: 18.6234\n",
      "INFO:tensorflow:global_step/sec: 0.581823\n",
      "INFO:tensorflow:examples/sec: 18.6183\n",
      "INFO:tensorflow:global_step/sec: 0.582331\n",
      "INFO:tensorflow:examples/sec: 18.6346\n",
      "INFO:tensorflow:global_step/sec: 0.58311\n",
      "INFO:tensorflow:examples/sec: 18.6595\n",
      "INFO:tensorflow:Saving checkpoints for 75000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.54426\n",
      "INFO:tensorflow:examples/sec: 17.4163\n",
      "INFO:tensorflow:global_step/sec: 0.584578\n",
      "INFO:tensorflow:examples/sec: 18.7065\n",
      "INFO:tensorflow:global_step/sec: 0.582432\n",
      "INFO:tensorflow:examples/sec: 18.6378\n",
      "INFO:tensorflow:global_step/sec: 0.582089\n",
      "INFO:tensorflow:examples/sec: 18.6269\n",
      "INFO:tensorflow:global_step/sec: 0.582169\n",
      "INFO:tensorflow:examples/sec: 18.6294\n",
      "INFO:tensorflow:global_step/sec: 0.581464\n",
      "INFO:tensorflow:examples/sec: 18.6069\n",
      "INFO:tensorflow:global_step/sec: 0.581411\n",
      "INFO:tensorflow:examples/sec: 18.6051\n",
      "INFO:tensorflow:global_step/sec: 0.581595\n",
      "INFO:tensorflow:examples/sec: 18.611\n",
      "INFO:tensorflow:global_step/sec: 0.581538\n",
      "INFO:tensorflow:examples/sec: 18.6092\n",
      "INFO:tensorflow:global_step/sec: 0.582474\n",
      "INFO:tensorflow:examples/sec: 18.6392\n",
      "INFO:tensorflow:Saving checkpoints for 76000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.543464\n",
      "INFO:tensorflow:examples/sec: 17.3909\n",
      "INFO:tensorflow:global_step/sec: 0.583795\n",
      "INFO:tensorflow:examples/sec: 18.6815\n",
      "INFO:tensorflow:global_step/sec: 0.582398\n",
      "INFO:tensorflow:examples/sec: 18.6367\n",
      "INFO:tensorflow:global_step/sec: 0.582385\n",
      "INFO:tensorflow:examples/sec: 18.6363\n",
      "INFO:tensorflow:global_step/sec: 0.582299\n",
      "INFO:tensorflow:examples/sec: 18.6336\n",
      "INFO:tensorflow:global_step/sec: 0.582196\n",
      "INFO:tensorflow:examples/sec: 18.6303\n",
      "INFO:tensorflow:global_step/sec: 0.581845\n",
      "INFO:tensorflow:examples/sec: 18.619\n",
      "INFO:tensorflow:global_step/sec: 0.581961\n",
      "INFO:tensorflow:examples/sec: 18.6228\n",
      "INFO:tensorflow:global_step/sec: 0.582199\n",
      "INFO:tensorflow:examples/sec: 18.6304\n",
      "INFO:tensorflow:global_step/sec: 0.581628\n",
      "INFO:tensorflow:examples/sec: 18.6121\n",
      "INFO:tensorflow:Saving checkpoints for 77000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.542848\n",
      "INFO:tensorflow:examples/sec: 17.3711\n",
      "INFO:tensorflow:global_step/sec: 0.58376\n",
      "INFO:tensorflow:examples/sec: 18.6803\n",
      "INFO:tensorflow:global_step/sec: 0.58197\n",
      "INFO:tensorflow:examples/sec: 18.623\n",
      "INFO:tensorflow:global_step/sec: 0.582042\n",
      "INFO:tensorflow:examples/sec: 18.6253\n",
      "INFO:tensorflow:global_step/sec: 0.581838\n",
      "INFO:tensorflow:examples/sec: 18.6188\n",
      "INFO:tensorflow:global_step/sec: 0.581702\n",
      "INFO:tensorflow:examples/sec: 18.6145\n",
      "INFO:tensorflow:global_step/sec: 0.582375\n",
      "INFO:tensorflow:examples/sec: 18.636\n",
      "INFO:tensorflow:global_step/sec: 0.582019\n",
      "INFO:tensorflow:examples/sec: 18.6246\n",
      "INFO:tensorflow:global_step/sec: 0.581308\n",
      "INFO:tensorflow:examples/sec: 18.6019\n",
      "INFO:tensorflow:global_step/sec: 0.581253\n",
      "INFO:tensorflow:examples/sec: 18.6001\n",
      "INFO:tensorflow:Saving checkpoints for 78000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.543382\n",
      "INFO:tensorflow:examples/sec: 17.3882\n",
      "INFO:tensorflow:global_step/sec: 0.583683\n",
      "INFO:tensorflow:examples/sec: 18.6779\n",
      "INFO:tensorflow:global_step/sec: 0.581847\n",
      "INFO:tensorflow:examples/sec: 18.6191\n",
      "INFO:tensorflow:global_step/sec: 0.581993\n",
      "INFO:tensorflow:examples/sec: 18.6238\n",
      "INFO:tensorflow:global_step/sec: 0.581688\n",
      "INFO:tensorflow:examples/sec: 18.614\n",
      "INFO:tensorflow:global_step/sec: 0.581258\n",
      "INFO:tensorflow:examples/sec: 18.6003\n",
      "INFO:tensorflow:global_step/sec: 0.58214\n",
      "INFO:tensorflow:examples/sec: 18.6285\n",
      "INFO:tensorflow:global_step/sec: 0.581299\n",
      "INFO:tensorflow:examples/sec: 18.6016\n",
      "INFO:tensorflow:global_step/sec: 0.581294\n",
      "INFO:tensorflow:examples/sec: 18.6014\n",
      "INFO:tensorflow:global_step/sec: 0.582214\n",
      "INFO:tensorflow:examples/sec: 18.6308\n",
      "INFO:tensorflow:Saving checkpoints for 79000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.543627\n",
      "INFO:tensorflow:examples/sec: 17.3961\n",
      "INFO:tensorflow:global_step/sec: 0.582544\n",
      "INFO:tensorflow:examples/sec: 18.6414\n",
      "INFO:tensorflow:global_step/sec: 0.582173\n",
      "INFO:tensorflow:examples/sec: 18.6295\n",
      "INFO:tensorflow:global_step/sec: 0.581272\n",
      "INFO:tensorflow:examples/sec: 18.6007\n",
      "INFO:tensorflow:global_step/sec: 0.580764\n",
      "INFO:tensorflow:examples/sec: 18.5845\n",
      "INFO:tensorflow:global_step/sec: 0.581989\n",
      "INFO:tensorflow:examples/sec: 18.6236\n",
      "INFO:tensorflow:global_step/sec: 0.582493\n",
      "INFO:tensorflow:examples/sec: 18.6398\n",
      "INFO:tensorflow:global_step/sec: 0.58218\n",
      "INFO:tensorflow:examples/sec: 18.6298\n",
      "INFO:tensorflow:global_step/sec: 0.581977\n",
      "INFO:tensorflow:examples/sec: 18.6233\n",
      "INFO:tensorflow:global_step/sec: 0.581336\n",
      "INFO:tensorflow:examples/sec: 18.6027\n",
      "INFO:tensorflow:Saving checkpoints for 80000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.543891\n",
      "INFO:tensorflow:examples/sec: 17.4045\n",
      "INFO:tensorflow:global_step/sec: 0.584109\n",
      "INFO:tensorflow:examples/sec: 18.6915\n",
      "INFO:tensorflow:global_step/sec: 0.582184\n",
      "INFO:tensorflow:examples/sec: 18.6299\n",
      "INFO:tensorflow:global_step/sec: 0.581993\n",
      "INFO:tensorflow:examples/sec: 18.6238\n",
      "INFO:tensorflow:global_step/sec: 0.580936\n",
      "INFO:tensorflow:examples/sec: 18.59\n",
      "INFO:tensorflow:global_step/sec: 0.580478\n",
      "INFO:tensorflow:examples/sec: 18.5753\n",
      "INFO:tensorflow:global_step/sec: 0.581319\n",
      "INFO:tensorflow:examples/sec: 18.6022\n",
      "INFO:tensorflow:global_step/sec: 0.580315\n",
      "INFO:tensorflow:examples/sec: 18.5701\n",
      "INFO:tensorflow:global_step/sec: 0.580535\n",
      "INFO:tensorflow:examples/sec: 18.5771\n",
      "INFO:tensorflow:global_step/sec: 0.581433\n",
      "INFO:tensorflow:examples/sec: 18.6059\n",
      "INFO:tensorflow:Saving checkpoints for 81000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.543323\n",
      "INFO:tensorflow:examples/sec: 17.3863\n",
      "INFO:tensorflow:global_step/sec: 0.58314\n",
      "INFO:tensorflow:examples/sec: 18.6605\n",
      "INFO:tensorflow:global_step/sec: 0.581199\n",
      "INFO:tensorflow:examples/sec: 18.5984\n",
      "INFO:tensorflow:global_step/sec: 0.581469\n",
      "INFO:tensorflow:examples/sec: 18.607\n",
      "INFO:tensorflow:global_step/sec: 0.581279\n",
      "INFO:tensorflow:examples/sec: 18.6009\n",
      "INFO:tensorflow:global_step/sec: 0.581052\n",
      "INFO:tensorflow:examples/sec: 18.5937\n",
      "INFO:tensorflow:global_step/sec: 0.580456\n",
      "INFO:tensorflow:examples/sec: 18.5746\n",
      "INFO:tensorflow:global_step/sec: 0.581244\n",
      "INFO:tensorflow:examples/sec: 18.5998\n",
      "INFO:tensorflow:global_step/sec: 0.580334\n",
      "INFO:tensorflow:examples/sec: 18.5707\n",
      "INFO:tensorflow:global_step/sec: 0.580121\n",
      "INFO:tensorflow:examples/sec: 18.5639\n",
      "INFO:tensorflow:Saving checkpoints for 82000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.543942\n",
      "INFO:tensorflow:examples/sec: 17.4062\n",
      "INFO:tensorflow:global_step/sec: 0.582184\n",
      "INFO:tensorflow:examples/sec: 18.6299\n",
      "INFO:tensorflow:global_step/sec: 0.581756\n",
      "INFO:tensorflow:examples/sec: 18.6162\n",
      "INFO:tensorflow:global_step/sec: 0.580749\n",
      "INFO:tensorflow:examples/sec: 18.584\n",
      "INFO:tensorflow:global_step/sec: 0.580843\n",
      "INFO:tensorflow:examples/sec: 18.587\n",
      "INFO:tensorflow:global_step/sec: 0.580085\n",
      "INFO:tensorflow:examples/sec: 18.5627\n",
      "INFO:tensorflow:global_step/sec: 0.580537\n",
      "INFO:tensorflow:examples/sec: 18.5772\n",
      "INFO:tensorflow:global_step/sec: 0.580326\n",
      "INFO:tensorflow:examples/sec: 18.5704\n",
      "INFO:tensorflow:global_step/sec: 0.579879\n",
      "INFO:tensorflow:examples/sec: 18.5561\n",
      "INFO:tensorflow:global_step/sec: 0.579908\n",
      "INFO:tensorflow:examples/sec: 18.557\n",
      "INFO:tensorflow:Saving checkpoints for 83000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.542602\n",
      "INFO:tensorflow:examples/sec: 17.3633\n",
      "INFO:tensorflow:global_step/sec: 0.582249\n",
      "INFO:tensorflow:examples/sec: 18.632\n",
      "INFO:tensorflow:global_step/sec: 0.581051\n",
      "INFO:tensorflow:examples/sec: 18.5936\n",
      "INFO:tensorflow:global_step/sec: 0.580785\n",
      "INFO:tensorflow:examples/sec: 18.5851\n",
      "INFO:tensorflow:global_step/sec: 0.58076\n",
      "INFO:tensorflow:examples/sec: 18.5843\n",
      "INFO:tensorflow:global_step/sec: 0.579283\n",
      "INFO:tensorflow:examples/sec: 18.537\n",
      "INFO:tensorflow:global_step/sec: 0.58027\n",
      "INFO:tensorflow:examples/sec: 18.5686\n",
      "INFO:tensorflow:global_step/sec: 0.58034\n",
      "INFO:tensorflow:examples/sec: 18.5709\n",
      "INFO:tensorflow:global_step/sec: 0.580568\n",
      "INFO:tensorflow:examples/sec: 18.5782\n",
      "INFO:tensorflow:global_step/sec: 0.580496\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:examples/sec: 18.5759\n",
      "INFO:tensorflow:Saving checkpoints for 84000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.542719\n",
      "INFO:tensorflow:examples/sec: 17.367\n",
      "INFO:tensorflow:global_step/sec: 0.582159\n",
      "INFO:tensorflow:examples/sec: 18.6291\n",
      "INFO:tensorflow:global_step/sec: 0.580971\n",
      "INFO:tensorflow:examples/sec: 18.5911\n",
      "INFO:tensorflow:global_step/sec: 0.580322\n",
      "INFO:tensorflow:examples/sec: 18.5703\n",
      "INFO:tensorflow:global_step/sec: 0.580638\n",
      "INFO:tensorflow:examples/sec: 18.5804\n",
      "INFO:tensorflow:global_step/sec: 0.58048\n",
      "INFO:tensorflow:examples/sec: 18.5754\n",
      "INFO:tensorflow:global_step/sec: 0.580264\n",
      "INFO:tensorflow:examples/sec: 18.5685\n",
      "INFO:tensorflow:global_step/sec: 0.58042\n",
      "INFO:tensorflow:examples/sec: 18.5734\n",
      "INFO:tensorflow:global_step/sec: 0.579663\n",
      "INFO:tensorflow:examples/sec: 18.5492\n",
      "INFO:tensorflow:global_step/sec: 0.580366\n",
      "INFO:tensorflow:examples/sec: 18.5717\n",
      "INFO:tensorflow:Saving checkpoints for 85000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.542645\n",
      "INFO:tensorflow:examples/sec: 17.3646\n",
      "INFO:tensorflow:global_step/sec: 0.582208\n",
      "INFO:tensorflow:examples/sec: 18.6307\n",
      "INFO:tensorflow:global_step/sec: 0.581076\n",
      "INFO:tensorflow:examples/sec: 18.5944\n",
      "INFO:tensorflow:global_step/sec: 0.580478\n",
      "INFO:tensorflow:examples/sec: 18.5753\n",
      "INFO:tensorflow:global_step/sec: 0.580505\n",
      "INFO:tensorflow:examples/sec: 18.5761\n",
      "INFO:tensorflow:global_step/sec: 0.580573\n",
      "INFO:tensorflow:examples/sec: 18.5783\n",
      "INFO:tensorflow:global_step/sec: 0.580072\n",
      "INFO:tensorflow:examples/sec: 18.5623\n",
      "INFO:tensorflow:global_step/sec: 0.580246\n",
      "INFO:tensorflow:examples/sec: 18.5679\n",
      "INFO:tensorflow:global_step/sec: 0.580516\n",
      "INFO:tensorflow:examples/sec: 18.5765\n",
      "INFO:tensorflow:global_step/sec: 0.580089\n",
      "INFO:tensorflow:examples/sec: 18.5629\n",
      "INFO:tensorflow:Saving checkpoints for 86000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.542943\n",
      "INFO:tensorflow:examples/sec: 17.3742\n",
      "INFO:tensorflow:global_step/sec: 0.581883\n",
      "INFO:tensorflow:examples/sec: 18.6203\n",
      "INFO:tensorflow:global_step/sec: 0.580624\n",
      "INFO:tensorflow:examples/sec: 18.58\n",
      "INFO:tensorflow:global_step/sec: 0.580668\n",
      "INFO:tensorflow:examples/sec: 18.5814\n",
      "INFO:tensorflow:global_step/sec: 0.580778\n",
      "INFO:tensorflow:examples/sec: 18.5849\n",
      "INFO:tensorflow:global_step/sec: 0.580386\n",
      "INFO:tensorflow:examples/sec: 18.5724\n",
      "INFO:tensorflow:global_step/sec: 0.580078\n",
      "INFO:tensorflow:examples/sec: 18.5625\n",
      "INFO:tensorflow:global_step/sec: 0.580162\n",
      "INFO:tensorflow:examples/sec: 18.5652\n",
      "INFO:tensorflow:global_step/sec: 0.580089\n",
      "INFO:tensorflow:examples/sec: 18.5629\n",
      "INFO:tensorflow:global_step/sec: 0.580276\n",
      "INFO:tensorflow:examples/sec: 18.5688\n",
      "INFO:tensorflow:Saving checkpoints for 87000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.541736\n",
      "INFO:tensorflow:examples/sec: 17.3356\n",
      "INFO:tensorflow:global_step/sec: 0.583393\n",
      "INFO:tensorflow:examples/sec: 18.6686\n",
      "INFO:tensorflow:global_step/sec: 0.581025\n",
      "INFO:tensorflow:examples/sec: 18.5928\n",
      "INFO:tensorflow:global_step/sec: 0.580978\n",
      "INFO:tensorflow:examples/sec: 18.5913\n",
      "INFO:tensorflow:global_step/sec: 0.580945\n",
      "INFO:tensorflow:examples/sec: 18.5903\n",
      "INFO:tensorflow:global_step/sec: 0.581034\n",
      "INFO:tensorflow:examples/sec: 18.5931\n",
      "INFO:tensorflow:global_step/sec: 0.581339\n",
      "INFO:tensorflow:examples/sec: 18.6028\n",
      "INFO:tensorflow:global_step/sec: 0.580132\n",
      "INFO:tensorflow:examples/sec: 18.5642\n",
      "INFO:tensorflow:global_step/sec: 0.580622\n",
      "INFO:tensorflow:examples/sec: 18.5799\n",
      "INFO:tensorflow:global_step/sec: 0.580312\n",
      "INFO:tensorflow:examples/sec: 18.57\n",
      "INFO:tensorflow:Saving checkpoints for 88000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.542611\n",
      "INFO:tensorflow:examples/sec: 17.3635\n",
      "INFO:tensorflow:global_step/sec: 0.582087\n",
      "INFO:tensorflow:examples/sec: 18.6268\n",
      "INFO:tensorflow:global_step/sec: 0.580643\n",
      "INFO:tensorflow:examples/sec: 18.5806\n",
      "INFO:tensorflow:global_step/sec: 0.580957\n",
      "INFO:tensorflow:examples/sec: 18.5906\n",
      "INFO:tensorflow:global_step/sec: 0.580966\n",
      "INFO:tensorflow:examples/sec: 18.5909\n",
      "INFO:tensorflow:global_step/sec: 0.580882\n",
      "INFO:tensorflow:examples/sec: 18.5882\n",
      "INFO:tensorflow:global_step/sec: 0.580324\n",
      "INFO:tensorflow:examples/sec: 18.5704\n",
      "INFO:tensorflow:global_step/sec: 0.580239\n",
      "INFO:tensorflow:examples/sec: 18.5676\n",
      "INFO:tensorflow:global_step/sec: 0.580809\n",
      "INFO:tensorflow:examples/sec: 18.5859\n",
      "INFO:tensorflow:global_step/sec: 0.580166\n",
      "INFO:tensorflow:examples/sec: 18.5653\n",
      "INFO:tensorflow:Saving checkpoints for 89000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.542227\n",
      "INFO:tensorflow:examples/sec: 17.3513\n",
      "INFO:tensorflow:global_step/sec: 0.58274\n",
      "INFO:tensorflow:examples/sec: 18.6477\n",
      "INFO:tensorflow:global_step/sec: 0.58104\n",
      "INFO:tensorflow:examples/sec: 18.5933\n",
      "INFO:tensorflow:global_step/sec: 0.579956\n",
      "INFO:tensorflow:examples/sec: 18.5586\n",
      "INFO:tensorflow:global_step/sec: 0.580297\n",
      "INFO:tensorflow:examples/sec: 18.5695\n",
      "INFO:tensorflow:global_step/sec: 0.580391\n",
      "INFO:tensorflow:examples/sec: 18.5725\n",
      "INFO:tensorflow:global_step/sec: 0.58012\n",
      "INFO:tensorflow:examples/sec: 18.5638\n",
      "INFO:tensorflow:global_step/sec: 0.579863\n",
      "INFO:tensorflow:examples/sec: 18.5556\n",
      "INFO:tensorflow:global_step/sec: 0.579918\n",
      "INFO:tensorflow:examples/sec: 18.5574\n",
      "INFO:tensorflow:global_step/sec: 0.580017\n",
      "INFO:tensorflow:examples/sec: 18.5606\n",
      "INFO:tensorflow:Saving checkpoints for 90000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.542242\n",
      "INFO:tensorflow:examples/sec: 17.3517\n",
      "INFO:tensorflow:global_step/sec: 0.581348\n",
      "INFO:tensorflow:examples/sec: 18.6031\n",
      "INFO:tensorflow:global_step/sec: 0.580393\n",
      "INFO:tensorflow:examples/sec: 18.5726\n",
      "INFO:tensorflow:global_step/sec: 0.580541\n",
      "INFO:tensorflow:examples/sec: 18.5773\n",
      "INFO:tensorflow:global_step/sec: 0.580084\n",
      "INFO:tensorflow:examples/sec: 18.5627\n",
      "INFO:tensorflow:global_step/sec: 0.58039\n",
      "INFO:tensorflow:examples/sec: 18.5725\n",
      "INFO:tensorflow:global_step/sec: 0.580028\n",
      "INFO:tensorflow:examples/sec: 18.5609\n",
      "INFO:tensorflow:global_step/sec: 0.580271\n",
      "INFO:tensorflow:examples/sec: 18.5687\n",
      "INFO:tensorflow:global_step/sec: 0.579846\n",
      "INFO:tensorflow:examples/sec: 18.5551\n",
      "INFO:tensorflow:global_step/sec: 0.57999\n",
      "INFO:tensorflow:examples/sec: 18.5597\n",
      "INFO:tensorflow:Saving checkpoints for 91000 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 0.542019\n",
      "INFO:tensorflow:examples/sec: 17.3446\n",
      "INFO:tensorflow:global_step/sec: 0.580995\n",
      "INFO:tensorflow:examples/sec: 18.5918\n",
      "INFO:tensorflow:global_step/sec: 0.58078\n",
      "INFO:tensorflow:examples/sec: 18.5849\n",
      "INFO:tensorflow:global_step/sec: 0.579794\n",
      "INFO:tensorflow:examples/sec: 18.5534\n",
      "INFO:tensorflow:global_step/sec: 0.579773\n",
      "INFO:tensorflow:examples/sec: 18.5527\n",
      "INFO:tensorflow:global_step/sec: 0.58037\n",
      "INFO:tensorflow:examples/sec: 18.5719\n",
      "INFO:tensorflow:global_step/sec: 0.580166\n",
      "INFO:tensorflow:examples/sec: 18.5653\n",
      "INFO:tensorflow:global_step/sec: 0.580315\n",
      "INFO:tensorflow:examples/sec: 18.5701\n",
      "INFO:tensorflow:global_step/sec: 0.580474\n",
      "INFO:tensorflow:examples/sec: 18.5752\n",
      "INFO:tensorflow:Saving checkpoints for 91836 into model_repo/outputs/model.ckpt.\n",
      "INFO:tensorflow:Loss for final step: 0.0033209533.\n",
      "INFO:tensorflow:training_loop marked as finished\n",
      "***** Finished training at 2019-03-16 06:35:17.988789 *****\n"
     ]
    }
   ],
   "source": [
    "\"\"\"\n",
    "Note: You might see a message 'Running train on CPU'. \n",
    "This really just means that it's running on something other than a Cloud TPU, which includes a GPU.\n",
    "\"\"\"\n",
    "\n",
    "# Train the model.\n",
    "print('Please wait...')\n",
    "train_features = run_classifier.convert_examples_to_features(\n",
    "    train_examples, label_list, MAX_SEQ_LENGTH, tokenizer)\n",
    "print('***** Started training at {} *****'.format(datetime.datetime.now()))\n",
    "print('  Num examples = {}'.format(len(train_examples)))\n",
    "print('  Batch size = {}'.format(TRAIN_BATCH_SIZE))\n",
    "tf.logging.info(\"  Num steps = %d\", num_train_steps)\n",
    "train_input_fn = run_classifier.input_fn_builder(\n",
    "    features=train_features,\n",
    "    seq_length=MAX_SEQ_LENGTH,\n",
    "    is_training=True,\n",
    "    drop_remainder=True)\n",
    "estimator.train(input_fn=train_input_fn, max_steps=num_train_steps)\n",
    "print('***** Finished training at {} *****'.format(datetime.datetime.now()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "_uuid": "58fb06968c09872ccd74b44e07b4a6e932beb6df",
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "There is a weird bug in original code.\n",
    "When predicting, estimator returns an empty dict {}, without batch_size.\n",
    "I redefine input_fn_builder and hardcode batch_size, irnoring 'params' for now.\n",
    "\"\"\"\n",
    "\n",
    "def input_fn_builder(features, seq_length, is_training, drop_remainder):\n",
    "    \"\"\"Creates an `input_fn` closure to be passed to TPUEstimator.\"\"\"\n",
    "    all_input_ids = []\n",
    "    all_input_mask = []\n",
    "    all_segment_ids = []\n",
    "    all_label_ids = []\n",
    "\n",
    "    for feature in features:\n",
    "        all_input_ids.append(feature.input_ids)\n",
    "        all_input_mask.append(feature.input_mask)\n",
    "        all_segment_ids.append(feature.segment_ids)\n",
    "        all_label_ids.append(feature.label_id)\n",
    "\n",
    "    def input_fn(params):\n",
    "        \"\"\"The actual input function.\"\"\"\n",
    "        print(params)\n",
    "        batch_size = 32\n",
    "\n",
    "        num_examples = len(features)\n",
    "\n",
    "        d = tf.data.Dataset.from_tensor_slices({\n",
    "            \"input_ids\":\n",
    "                tf.constant(\n",
    "                    all_input_ids, shape=[num_examples, seq_length],\n",
    "                    dtype=tf.int32),\n",
    "            \"input_mask\":\n",
    "                tf.constant(\n",
    "                    all_input_mask,\n",
    "                    shape=[num_examples, seq_length],\n",
    "                    dtype=tf.int32),\n",
    "            \"segment_ids\":\n",
    "                tf.constant(\n",
    "                    all_segment_ids,\n",
    "                    shape=[num_examples, seq_length],\n",
    "                    dtype=tf.int32),\n",
    "            \"label_ids\":\n",
    "                tf.constant(all_label_ids, shape=[num_examples], dtype=tf.int32),\n",
    "        })\n",
    "\n",
    "        if is_training:\n",
    "            d = d.repeat()\n",
    "            d = d.shuffle(buffer_size=100)\n",
    "\n",
    "        d = d.batch(batch_size=batch_size, drop_remainder=drop_remainder)\n",
    "        return d\n",
    "\n",
    "    return input_fn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "_uuid": "20a827baca921e1b48ac3de20422e6d384e2e450"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Writing example 0 of 130613\n",
      "INFO:tensorflow:*** Example ***\n",
      "INFO:tensorflow:guid: test\n",
      "INFO:tensorflow:tokens: [CLS] what is the most effective classroom management skill / technique to create a good learning environment ? [SEP]\n",
      "INFO:tensorflow:input_ids: 101 2054 2003 1996 2087 4621 9823 2968 8066 1013 6028 2000 3443 1037 2204 4083 4044 1029 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
      "INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
      "INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
      "INFO:tensorflow:label: 0 (id = 0)\n",
      "INFO:tensorflow:*** Example ***\n",
      "INFO:tensorflow:guid: test\n",
      "INFO:tensorflow:tokens: [CLS] can i study abroad after 10th class from bangladesh ? [SEP]\n",
      "INFO:tensorflow:input_ids: 101 2064 1045 2817 6917 2044 6049 2465 2013 7269 1029 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
      "INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
      "INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
      "INFO:tensorflow:label: 0 (id = 0)\n",
      "INFO:tensorflow:*** Example ***\n",
      "INFO:tensorflow:guid: test\n",
      "INFO:tensorflow:tokens: [CLS] how can i make friends as a college junior ? [SEP]\n",
      "INFO:tensorflow:input_ids: 101 2129 2064 1045 2191 2814 2004 1037 2267 3502 1029 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
      "INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
      "INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
      "INFO:tensorflow:label: 0 (id = 0)\n",
      "INFO:tensorflow:*** Example ***\n",
      "INFO:tensorflow:guid: test\n",
      "INFO:tensorflow:tokens: [CLS] how do i download free ap ##k mine ##craft : pocket edition for ios ( iphone ) ? [SEP]\n",
      "INFO:tensorflow:input_ids: 101 2129 2079 1045 8816 2489 9706 2243 3067 10419 1024 4979 3179 2005 16380 1006 18059 1007 1029 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
      "INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
      "INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
      "INFO:tensorflow:label: 0 (id = 0)\n",
      "INFO:tensorflow:*** Example ***\n",
      "INFO:tensorflow:guid: test\n",
      "INFO:tensorflow:tokens: [CLS] like ku ##vera , is \" grow ##w \" also a free online investment platform where i can invest in direct mutual funds ? [SEP]\n",
      "INFO:tensorflow:input_ids: 101 2066 13970 26061 1010 2003 1000 4982 2860 1000 2036 1037 2489 3784 5211 4132 2073 1045 2064 15697 1999 3622 8203 5029 1029 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
      "INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
      "INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
      "INFO:tensorflow:label: 0 (id = 0)\n",
      "INFO:tensorflow:Writing example 10000 of 130613\n",
      "INFO:tensorflow:Writing example 20000 of 130613\n",
      "INFO:tensorflow:Writing example 30000 of 130613\n",
      "INFO:tensorflow:Writing example 40000 of 130613\n",
      "INFO:tensorflow:Writing example 50000 of 130613\n",
      "INFO:tensorflow:Writing example 60000 of 130613\n",
      "INFO:tensorflow:Writing example 70000 of 130613\n",
      "INFO:tensorflow:Writing example 80000 of 130613\n",
      "INFO:tensorflow:Writing example 90000 of 130613\n",
      "INFO:tensorflow:Writing example 100000 of 130613\n",
      "INFO:tensorflow:Writing example 110000 of 130613\n",
      "INFO:tensorflow:Writing example 120000 of 130613\n",
      "INFO:tensorflow:Writing example 130000 of 130613\n"
     ]
    }
   ],
   "source": [
    "predict_examples = create_examples(test_lines, 'test')\n",
    "\n",
    "predict_features = run_classifier.convert_examples_to_features(\n",
    "    predict_examples, label_list, MAX_SEQ_LENGTH, tokenizer)\n",
    "\n",
    "predict_input_fn = input_fn_builder(\n",
    "    features=predict_features,\n",
    "    seq_length=MAX_SEQ_LENGTH,\n",
    "    is_training=False,\n",
    "    drop_remainder=False)\n",
    "\n",
    "result = estimator.predict(input_fn=predict_input_fn)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "_uuid": "ee2e686b144528d8eb5a2c50f98c52816a14c657"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "0it [00:00, ?it/s]\u001b[A"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{}\n",
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Running infer on CPU\n",
      "INFO:tensorflow:*** Features ***\n",
      "INFO:tensorflow:  name = input_ids, shape = (?, 128)\n",
      "INFO:tensorflow:  name = input_mask, shape = (?, 128)\n",
      "INFO:tensorflow:  name = label_ids, shape = (?,)\n",
      "INFO:tensorflow:  name = segment_ids, shape = (?, 128)\n",
      "INFO:tensorflow:**** Trainable Variables ****\n",
      "INFO:tensorflow:  name = bert/embeddings/word_embeddings:0, shape = (30522, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/embeddings/token_type_embeddings:0, shape = (2, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/embeddings/position_embeddings:0, shape = (512, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/embeddings/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/embeddings/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_0/attention/self/query/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_0/attention/self/query/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_0/attention/self/key/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_0/attention/self/key/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_0/attention/self/value/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_0/attention/self/value/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_0/attention/output/dense/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_0/attention/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_0/attention/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_0/attention/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_0/intermediate/dense/kernel:0, shape = (768, 3072), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_0/intermediate/dense/bias:0, shape = (3072,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_0/output/dense/kernel:0, shape = (3072, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_0/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_0/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_0/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_1/attention/self/query/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_1/attention/self/query/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_1/attention/self/key/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_1/attention/self/key/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_1/attention/self/value/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_1/attention/self/value/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_1/attention/output/dense/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_1/attention/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_1/attention/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_1/attention/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_1/intermediate/dense/kernel:0, shape = (768, 3072), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_1/intermediate/dense/bias:0, shape = (3072,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_1/output/dense/kernel:0, shape = (3072, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_1/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_1/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_1/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_2/attention/self/query/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_2/attention/self/query/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_2/attention/self/key/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_2/attention/self/key/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_2/attention/self/value/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_2/attention/self/value/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_2/attention/output/dense/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_2/attention/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_2/attention/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_2/attention/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_2/intermediate/dense/kernel:0, shape = (768, 3072), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_2/intermediate/dense/bias:0, shape = (3072,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_2/output/dense/kernel:0, shape = (3072, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_2/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_2/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_2/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_3/attention/self/query/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_3/attention/self/query/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_3/attention/self/key/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_3/attention/self/key/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_3/attention/self/value/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_3/attention/self/value/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_3/attention/output/dense/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_3/attention/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_3/attention/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_3/attention/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_3/intermediate/dense/kernel:0, shape = (768, 3072), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_3/intermediate/dense/bias:0, shape = (3072,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_3/output/dense/kernel:0, shape = (3072, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_3/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_3/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_3/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_4/attention/self/query/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_4/attention/self/query/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_4/attention/self/key/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:  name = bert/encoder/layer_4/attention/self/key/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_4/attention/self/value/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_4/attention/self/value/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_4/attention/output/dense/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_4/attention/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_4/attention/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_4/attention/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_4/intermediate/dense/kernel:0, shape = (768, 3072), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_4/intermediate/dense/bias:0, shape = (3072,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_4/output/dense/kernel:0, shape = (3072, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_4/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_4/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_4/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_5/attention/self/query/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_5/attention/self/query/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_5/attention/self/key/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_5/attention/self/key/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_5/attention/self/value/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_5/attention/self/value/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_5/attention/output/dense/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_5/attention/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_5/attention/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_5/attention/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_5/intermediate/dense/kernel:0, shape = (768, 3072), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_5/intermediate/dense/bias:0, shape = (3072,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_5/output/dense/kernel:0, shape = (3072, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_5/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_5/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_5/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_6/attention/self/query/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_6/attention/self/query/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_6/attention/self/key/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_6/attention/self/key/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_6/attention/self/value/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_6/attention/self/value/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_6/attention/output/dense/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_6/attention/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_6/attention/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_6/attention/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_6/intermediate/dense/kernel:0, shape = (768, 3072), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_6/intermediate/dense/bias:0, shape = (3072,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_6/output/dense/kernel:0, shape = (3072, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_6/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_6/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_6/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_7/attention/self/query/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_7/attention/self/query/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_7/attention/self/key/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_7/attention/self/key/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_7/attention/self/value/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_7/attention/self/value/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_7/attention/output/dense/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_7/attention/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_7/attention/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_7/attention/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_7/intermediate/dense/kernel:0, shape = (768, 3072), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_7/intermediate/dense/bias:0, shape = (3072,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_7/output/dense/kernel:0, shape = (3072, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_7/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_7/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_7/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_8/attention/self/query/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_8/attention/self/query/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_8/attention/self/key/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_8/attention/self/key/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_8/attention/self/value/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_8/attention/self/value/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_8/attention/output/dense/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_8/attention/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_8/attention/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_8/attention/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_8/intermediate/dense/kernel:0, shape = (768, 3072), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_8/intermediate/dense/bias:0, shape = (3072,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_8/output/dense/kernel:0, shape = (3072, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_8/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:  name = bert/encoder/layer_8/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_8/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_9/attention/self/query/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_9/attention/self/query/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_9/attention/self/key/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_9/attention/self/key/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_9/attention/self/value/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_9/attention/self/value/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_9/attention/output/dense/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_9/attention/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_9/attention/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_9/attention/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_9/intermediate/dense/kernel:0, shape = (768, 3072), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_9/intermediate/dense/bias:0, shape = (3072,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_9/output/dense/kernel:0, shape = (3072, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_9/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_9/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_9/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_10/attention/self/query/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_10/attention/self/query/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_10/attention/self/key/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_10/attention/self/key/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_10/attention/self/value/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_10/attention/self/value/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_10/attention/output/dense/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_10/attention/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_10/attention/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_10/attention/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_10/intermediate/dense/kernel:0, shape = (768, 3072), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_10/intermediate/dense/bias:0, shape = (3072,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_10/output/dense/kernel:0, shape = (3072, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_10/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_10/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_10/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_11/attention/self/query/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_11/attention/self/query/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_11/attention/self/key/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_11/attention/self/key/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_11/attention/self/value/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_11/attention/self/value/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_11/attention/output/dense/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_11/attention/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_11/attention/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_11/attention/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_11/intermediate/dense/kernel:0, shape = (768, 3072), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_11/intermediate/dense/bias:0, shape = (3072,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_11/output/dense/kernel:0, shape = (3072, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_11/output/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_11/output/LayerNorm/beta:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/encoder/layer_11/output/LayerNorm/gamma:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/pooler/dense/kernel:0, shape = (768, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = bert/pooler/dense/bias:0, shape = (768,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = output_weights:0, shape = (2, 768), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:  name = output_bias:0, shape = (2,), *INIT_FROM_CKPT*\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Restoring parameters from model_repo/outputs/model.ckpt-91836\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "1it [01:12, 72.13s/it]\u001b[A\n",
      "33it [01:12, 50.50s/it]\u001b[A\n",
      "65it [01:13, 35.35s/it]\u001b[A\n",
      "97it [01:13, 24.75s/it]\u001b[A\n",
      "129it [01:14, 17.33s/it]\u001b[A\n",
      "161it [01:15, 12.14s/it]\u001b[A\n",
      "193it [01:15,  8.50s/it]\u001b[A\n",
      "225it [01:16,  5.96s/it]\u001b[A\n",
      "257it [01:16,  4.18s/it]\u001b[A\n",
      "289it [01:17,  2.93s/it]\u001b[A\n",
      "321it [01:18,  2.06s/it]\u001b[A\n",
      "353it [01:18,  1.44s/it]\u001b[A\n",
      "385it [01:19,  1.02s/it]\u001b[A\n",
      "417it [01:19,  1.39it/s]\u001b[A\n",
      "449it [01:20,  1.97it/s]\u001b[A\n",
      "481it [01:21,  2.77it/s]\u001b[A\n",
      "513it [01:21,  3.87it/s]\u001b[A\n",
      "545it [01:22,  5.36it/s]\u001b[A\n",
      "577it [01:22,  7.35it/s]\u001b[A\n",
      "609it [01:23,  9.92it/s]\u001b[A\n",
      "641it [01:24, 13.12it/s]\u001b[A\n",
      "673it [01:24, 16.96it/s]\u001b[A\n",
      "705it [01:25, 21.31it/s]\u001b[A\n",
      "737it [01:25, 25.99it/s]\u001b[A\n",
      "769it [01:26, 30.73it/s]\u001b[A\n",
      "801it [01:27, 35.26it/s]\u001b[A\n",
      "833it [01:27, 39.34it/s]\u001b[A\n",
      "865it [01:28, 42.54it/s]\u001b[A\n",
      "897it [01:28, 45.34it/s]\u001b[A\n",
      "929it [01:29, 47.28it/s]\u001b[A\n",
      "961it [01:30, 48.98it/s]\u001b[A\n",
      "993it [01:30, 50.21it/s]\u001b[A\n",
      "1025it [01:31, 51.07it/s]\u001b[A\n",
      "1057it [01:31, 51.75it/s]\u001b[A\n",
      "1089it [01:32, 52.37it/s]\u001b[A\n",
      "1121it [01:33, 52.43it/s]\u001b[A\n",
      "1153it [01:33, 52.72it/s]\u001b[A\n",
      "1185it [01:34, 52.68it/s]\u001b[A\n",
      "1217it [01:34, 52.69it/s]\u001b[A\n",
      "1249it [01:35, 52.88it/s]\u001b[A\n",
      "1281it [01:36, 53.01it/s]\u001b[A\n",
      "1313it [01:36, 52.86it/s]\u001b[A\n",
      "1345it [01:37, 53.01it/s]\u001b[A\n",
      "1377it [01:37, 53.31it/s]\u001b[A\n",
      "1409it [01:38, 53.10it/s]\u001b[A\n",
      "1441it [01:39, 53.16it/s]\u001b[A\n",
      "1473it [01:39, 52.97it/s]\u001b[A\n",
      "1505it [01:40, 52.77it/s]\u001b[A\n",
      "1537it [01:41, 52.49it/s]\u001b[A\n",
      "1569it [01:41, 52.84it/s]\u001b[A\n",
      "1601it [01:42, 53.03it/s]\u001b[A\n",
      "1633it [01:42, 53.29it/s]\u001b[A\n",
      "1665it [01:43, 53.34it/s]\u001b[A\n",
      "1697it [01:44, 53.18it/s]\u001b[A\n",
      "1729it [01:44, 53.28it/s]\u001b[A\n",
      "1761it [01:45, 53.17it/s]\u001b[A\n",
      "1793it [01:45, 53.26it/s]\u001b[A\n",
      "1825it [01:46, 53.00it/s]\u001b[A\n",
      "1857it [01:47, 53.08it/s]\u001b[A\n",
      "1889it [01:47, 52.99it/s]\u001b[A\n",
      "1921it [01:48, 53.06it/s]\u001b[A\n",
      "1953it [01:48, 53.23it/s]\u001b[A\n",
      "1985it [01:49, 53.04it/s]\u001b[A\n",
      "2017it [01:50, 53.27it/s]\u001b[A\n",
      "2049it [01:50, 53.04it/s]\u001b[A\n",
      "2081it [01:51, 53.11it/s]\u001b[A\n",
      "2113it [01:51, 53.04it/s]\u001b[A\n",
      "2145it [01:52, 53.12it/s]\u001b[A\n",
      "2177it [01:53, 53.23it/s]\u001b[A\n",
      "2209it [01:53, 53.16it/s]\u001b[A\n",
      "2241it [01:54, 53.23it/s]\u001b[A\n",
      "2273it [01:54, 52.93it/s]\u001b[A\n",
      "2305it [01:55, 53.20it/s]\u001b[A\n",
      "2337it [01:56, 53.44it/s]\u001b[A\n",
      "2369it [01:56, 53.58it/s]\u001b[A\n",
      "2401it [01:57, 53.15it/s]\u001b[A\n",
      "2433it [01:57, 53.00it/s]\u001b[A\n",
      "2465it [01:58, 53.13it/s]\u001b[A\n",
      "2497it [01:59, 52.94it/s]\u001b[A\n",
      "2529it [01:59, 53.03it/s]\u001b[A\n",
      "2561it [02:00, 53.30it/s]\u001b[A\n",
      "2593it [02:00, 53.53it/s]\u001b[A\n",
      "2625it [02:01, 53.59it/s]\u001b[A\n",
      "2657it [02:02, 53.69it/s]\u001b[A\n",
      "2689it [02:02, 53.33it/s]\u001b[A\n",
      "2721it [02:03, 53.31it/s]\u001b[A\n",
      "2753it [02:03, 53.10it/s]\u001b[A\n",
      "2785it [02:04, 53.16it/s]\u001b[A\n",
      "2817it [02:05, 52.97it/s]\u001b[A\n",
      "2849it [02:05, 52.93it/s]\u001b[A\n",
      "2881it [02:06, 53.04it/s]\u001b[A\n",
      "2913it [02:06, 53.15it/s]\u001b[A\n",
      "2945it [02:07, 52.76it/s]\u001b[A\n",
      "2977it [02:08, 52.60it/s]\u001b[A\n",
      "3009it [02:08, 52.79it/s]\u001b[A\n",
      "3041it [02:09, 52.90it/s]\u001b[A\n",
      "3073it [02:09, 53.05it/s]\u001b[A\n",
      "3105it [02:10, 52.76it/s]\u001b[A\n",
      "3137it [02:11, 53.03it/s]\u001b[A\n",
      "3169it [02:11, 52.94it/s]\u001b[A\n",
      "3201it [02:12, 53.07it/s]\u001b[A\n",
      "3233it [02:12, 52.99it/s]\u001b[A\n",
      "3265it [02:13, 53.12it/s]\u001b[A\n",
      "3297it [02:14, 52.93it/s]\u001b[A\n",
      "3329it [02:14, 53.08it/s]\u001b[A\n",
      "3361it [02:15, 53.25it/s]\u001b[A\n",
      "3393it [02:15, 53.31it/s]\u001b[A\n",
      "3425it [02:16, 53.21it/s]\u001b[A\n",
      "3457it [02:17, 53.29it/s]\u001b[A\n",
      "3489it [02:17, 53.29it/s]\u001b[A\n",
      "3521it [02:18, 53.25it/s]\u001b[A\n",
      "3553it [02:18, 53.28it/s]\u001b[A\n",
      "3585it [02:19, 53.11it/s]\u001b[A\n",
      "3617it [02:20, 53.21it/s]\u001b[A\n",
      "3649it [02:20, 53.43it/s]\u001b[A\n",
      "3681it [02:21, 53.39it/s]\u001b[A\n",
      "3713it [02:21, 53.31it/s]\u001b[A\n",
      "3745it [02:22, 53.35it/s]\u001b[A\n",
      "3777it [02:23, 53.51it/s]\u001b[A\n",
      "3809it [02:23, 53.57it/s]\u001b[A\n",
      "3841it [02:24, 53.18it/s]\u001b[A\n",
      "3873it [02:24, 52.97it/s]\u001b[A\n",
      "3905it [02:25, 53.14it/s]\u001b[A\n",
      "3937it [02:26, 52.91it/s]\u001b[A\n",
      "3969it [02:26, 53.05it/s]\u001b[A\n",
      "4001it [02:27, 53.15it/s]\u001b[A\n",
      "4033it [02:27, 53.17it/s]\u001b[A\n",
      "4065it [02:28, 53.26it/s]\u001b[A\n",
      "4097it [02:29, 53.06it/s]\u001b[A\n",
      "4129it [02:29, 53.17it/s]\u001b[A\n",
      "4161it [02:30, 53.00it/s]\u001b[A\n",
      "4193it [02:30, 52.72it/s]\u001b[A\n",
      "4225it [02:31, 52.56it/s]\u001b[A\n",
      "4257it [02:32, 52.75it/s]\u001b[A\n",
      "4289it [02:32, 53.13it/s]\u001b[A\n",
      "4321it [02:33, 53.32it/s]\u001b[A\n",
      "4353it [02:33, 53.10it/s]\u001b[A\n",
      "4385it [02:34, 53.21it/s]\u001b[A\n",
      "4417it [02:35, 53.41it/s]\u001b[A\n",
      "4449it [02:35, 53.43it/s]\u001b[A\n",
      "4481it [02:36, 53.27it/s]\u001b[A\n",
      "4513it [02:36, 53.32it/s]\u001b[A\n",
      "4545it [02:37, 53.46it/s]\u001b[A\n",
      "4577it [02:38, 53.58it/s]\u001b[A\n",
      "4609it [02:38, 53.49it/s]\u001b[A\n",
      "4641it [02:39, 53.39it/s]\u001b[A\n",
      "4673it [02:39, 53.22it/s]\u001b[A\n",
      "4705it [02:40, 53.16it/s]\u001b[A\n",
      "4737it [02:41, 52.89it/s]\u001b[A\n",
      "4769it [02:41, 53.07it/s]\u001b[A\n",
      "4801it [02:42, 52.88it/s]\u001b[A\n",
      "4833it [02:43, 52.70it/s]\u001b[A\n",
      "4865it [02:43, 52.89it/s]\u001b[A\n",
      "4897it [02:44, 52.80it/s]\u001b[A\n",
      "4929it [02:44, 52.98it/s]\u001b[A\n",
      "4961it [02:45, 53.18it/s]\u001b[A\n",
      "4993it [02:46, 53.04it/s]\u001b[A\n",
      "5025it [02:46, 53.17it/s]\u001b[A\n",
      "5057it [02:47, 52.86it/s]\u001b[A\n",
      "5089it [02:47, 52.96it/s]\u001b[A\n",
      "5121it [02:48, 53.02it/s]\u001b[A\n",
      "5153it [02:49, 53.12it/s]\u001b[A\n",
      "5185it [02:49, 52.94it/s]\u001b[A\n",
      "5217it [02:50, 53.11it/s]\u001b[A\n",
      "5249it [02:50, 53.11it/s]\u001b[A\n",
      "5281it [02:51, 53.02it/s]\u001b[A\n",
      "5313it [02:52, 53.15it/s]\u001b[A\n",
      "5345it [02:52, 53.13it/s]\u001b[A\n",
      "5377it [02:53, 53.21it/s]\u001b[A\n",
      "5409it [02:53, 53.40it/s]\u001b[A\n",
      "5441it [02:54, 53.44it/s]\u001b[A\n",
      "5473it [02:55, 53.23it/s]\u001b[A\n",
      "5505it [02:55, 53.33it/s]\u001b[A\n",
      "5537it [02:56, 53.39it/s]\u001b[A\n",
      "5569it [02:56, 53.24it/s]\u001b[A\n",
      "5601it [02:57, 53.29it/s]\u001b[A\n",
      "5633it [02:58, 53.44it/s]\u001b[A\n",
      "5665it [02:58, 53.23it/s]\u001b[A\n",
      "5697it [02:59, 53.28it/s]\u001b[A\n",
      "5729it [02:59, 53.11it/s]\u001b[A\n",
      "5761it [03:00, 52.73it/s]\u001b[A\n",
      "5793it [03:01, 53.00it/s]\u001b[A\n",
      "5825it [03:01, 53.21it/s]\u001b[A\n",
      "5857it [03:02, 52.90it/s]\u001b[A\n",
      "5889it [03:02, 52.84it/s]\u001b[A\n",
      "5921it [03:03, 52.98it/s]\u001b[A\n",
      "5953it [03:04, 52.90it/s]\u001b[A\n",
      "5985it [03:04, 52.99it/s]\u001b[A\n",
      "6017it [03:05, 52.87it/s]\u001b[A\n",
      "6049it [03:05, 52.60it/s]\u001b[A\n",
      "6081it [03:06, 52.86it/s]\u001b[A\n",
      "6113it [03:07, 52.78it/s]\u001b[A\n",
      "6145it [03:07, 52.98it/s]\u001b[A\n",
      "6177it [03:08, 52.82it/s]\u001b[A\n",
      "6209it [03:08, 52.63it/s]\u001b[A\n",
      "6241it [03:09, 52.81it/s]\u001b[A\n",
      "6273it [03:10, 52.75it/s]\u001b[A\n",
      "6305it [03:10, 52.93it/s]\u001b[A\n",
      "6337it [03:11, 52.99it/s]\u001b[A\n",
      "6369it [03:11, 53.10it/s]\u001b[A\n",
      "6401it [03:12, 53.19it/s]\u001b[A\n",
      "6433it [03:13, 53.30it/s]\u001b[A\n",
      "6465it [03:13, 53.17it/s]\u001b[A\n",
      "6497it [03:14, 53.19it/s]\u001b[A\n",
      "6529it [03:14, 53.01it/s]\u001b[A\n",
      "6561it [03:15, 52.73it/s]\u001b[A\n",
      "6593it [03:16, 52.98it/s]\u001b[A\n",
      "6625it [03:16, 52.69it/s]\u001b[A\n",
      "6657it [03:17, 52.91it/s]\u001b[A\n",
      "6689it [03:18, 52.91it/s]\u001b[A\n",
      "6721it [03:18, 53.11it/s]\u001b[A\n",
      "6753it [03:19, 53.09it/s]\u001b[A\n",
      "6785it [03:19, 53.07it/s]\u001b[A\n",
      "6817it [03:20, 52.87it/s]\u001b[A\n",
      "6849it [03:21, 53.06it/s]\u001b[A\n",
      "6881it [03:21, 53.13it/s]\u001b[A\n",
      "6913it [03:22, 53.16it/s]\u001b[A\n",
      "6945it [03:22, 53.23it/s]\u001b[A\n",
      "6977it [03:23, 53.30it/s]\u001b[A\n",
      "7009it [03:24, 53.16it/s]\u001b[A\n",
      "7041it [03:24, 52.87it/s]\u001b[A\n",
      "7073it [03:25, 53.06it/s]\u001b[A\n",
      "7105it [03:25, 52.90it/s]\u001b[A\n",
      "7137it [03:26, 52.70it/s]\u001b[A\n",
      "7169it [03:27, 52.92it/s]\u001b[A\n",
      "7201it [03:27, 53.02it/s]\u001b[A\n",
      "7233it [03:28, 53.06it/s]\u001b[A\n",
      "7265it [03:28, 53.14it/s]\u001b[A\n",
      "7297it [03:29, 52.85it/s]\u001b[A\n",
      "7329it [03:30, 52.97it/s]\u001b[A\n",
      "7361it [03:30, 53.05it/s]\u001b[A\n",
      "7393it [03:31, 53.13it/s]\u001b[A\n",
      "7425it [03:31, 52.96it/s]\u001b[A\n",
      "7457it [03:32, 53.13it/s]\u001b[A\n",
      "7489it [03:33, 53.27it/s]\u001b[A\n",
      "7521it [03:33, 53.10it/s]\u001b[A\n",
      "7553it [03:34, 53.18it/s]\u001b[A\n",
      "7585it [03:34, 53.20it/s]\u001b[A\n",
      "7617it [03:35, 53.20it/s]\u001b[A\n",
      "7649it [03:36, 53.22it/s]\u001b[A\n",
      "7681it [03:36, 52.87it/s]\u001b[A\n",
      "7713it [03:37, 52.87it/s]\u001b[A\n",
      "7745it [03:37, 53.02it/s]\u001b[A\n",
      "7777it [03:38, 53.16it/s]\u001b[A\n",
      "7809it [03:39, 53.08it/s]\u001b[A\n",
      "7841it [03:39, 53.19it/s]\u001b[A\n",
      "7873it [03:40, 52.96it/s]\u001b[A\n",
      "7905it [03:40, 53.10it/s]\u001b[A\n",
      "7937it [03:41, 52.96it/s]\u001b[A\n",
      "7969it [03:42, 53.14it/s]\u001b[A\n",
      "8001it [03:42, 52.94it/s]\u001b[A\n",
      "8033it [03:43, 53.09it/s]\u001b[A\n",
      "8065it [03:43, 52.91it/s]\u001b[A\n",
      "8097it [03:44, 52.71it/s]\u001b[A\n",
      "8129it [03:45, 52.91it/s]\u001b[A\n",
      "8161it [03:45, 52.86it/s]\u001b[A\n",
      "8193it [03:46, 52.86it/s]\u001b[A\n",
      "8225it [03:46, 52.82it/s]\u001b[A\n",
      "8257it [03:47, 53.05it/s]\u001b[A\n",
      "8289it [03:48, 52.93it/s]\u001b[A\n",
      "8321it [03:48, 52.59it/s]\u001b[A\n",
      "8353it [03:49, 52.74it/s]\u001b[A\n",
      "8385it [03:50, 52.91it/s]\u001b[A\n",
      "8417it [03:50, 53.06it/s]\u001b[A\n",
      "8449it [03:51, 52.86it/s]\u001b[A\n",
      "8481it [03:51, 53.07it/s]\u001b[A\n",
      "8513it [03:52, 52.96it/s]\u001b[A\n",
      "8545it [03:53, 52.89it/s]\u001b[A\n",
      "8577it [03:53, 52.95it/s]\u001b[A\n",
      "8609it [03:54, 53.05it/s]\u001b[A\n",
      "8641it [03:54, 52.95it/s]\u001b[A\n",
      "8673it [03:55, 53.07it/s]\u001b[A\n",
      "8705it [03:56, 52.95it/s]\u001b[A\n",
      "8737it [03:56, 53.04it/s]\u001b[A\n",
      "8769it [03:57, 52.92it/s]\u001b[A\n",
      "8801it [03:57, 53.07it/s]\u001b[A\n",
      "8833it [03:58, 52.94it/s]\u001b[A\n",
      "8865it [03:59, 52.66it/s]\u001b[A\n",
      "8897it [03:59, 52.86it/s]\u001b[A\n",
      "8929it [04:00, 52.83it/s]\u001b[A\n",
      "8961it [04:00, 53.00it/s]\u001b[A\n",
      "8993it [04:01, 52.84it/s]\u001b[A\n",
      "9025it [04:02, 52.96it/s]\u001b[A\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "9057it [04:02, 52.75it/s]\u001b[A\n",
      "9089it [04:03, 52.68it/s]\u001b[A\n",
      "9121it [04:03, 52.41it/s]\u001b[A\n",
      "9153it [04:04, 52.43it/s]\u001b[A\n",
      "9185it [04:05, 52.78it/s]\u001b[A\n",
      "9217it [04:05, 52.70it/s]\u001b[A\n",
      "9249it [04:06, 52.91it/s]\u001b[A\n",
      "9281it [04:06, 52.87it/s]\u001b[A\n",
      "9313it [04:07, 52.58it/s]\u001b[A\n",
      "9345it [04:08, 52.41it/s]\u001b[A\n",
      "9377it [04:08, 52.76it/s]\u001b[A\n",
      "9409it [04:09, 52.61it/s]\u001b[A\n",
      "9441it [04:10, 52.52it/s]\u001b[A\n",
      "9473it [04:10, 52.38it/s]\u001b[A\n",
      "9505it [04:11, 52.32it/s]\u001b[A\n",
      "9537it [04:11, 52.65it/s]\u001b[A\n",
      "9569it [04:12, 52.59it/s]\u001b[A\n",
      "9601it [04:13, 52.78it/s]\u001b[A\n",
      "9633it [04:13, 52.80it/s]\u001b[A\n",
      "9665it [04:14, 52.94it/s]\u001b[A\n",
      "9697it [04:14, 52.87it/s]\u001b[A\n",
      "9729it [04:15, 52.64it/s]\u001b[A\n",
      "9761it [04:16, 52.52it/s]\u001b[A\n",
      "9793it [04:16, 52.36it/s]\u001b[A\n",
      "9825it [04:17, 52.69it/s]\u001b[A\n",
      "9857it [04:17, 52.51it/s]\u001b[A\n",
      "9889it [04:18, 52.80it/s]\u001b[A\n",
      "9921it [04:19, 52.60it/s]\u001b[A\n",
      "9953it [04:19, 52.66it/s]\u001b[A\n",
      "9985it [04:20, 52.86it/s]\u001b[A\n",
      "10017it [04:20, 53.12it/s]\u001b[A\n",
      "10049it [04:21, 52.92it/s]\u001b[A\n",
      "10081it [04:22, 53.11it/s]\u001b[A\n",
      "10113it [04:22, 52.93it/s]\u001b[A\n",
      "10145it [04:23, 53.11it/s]\u001b[A\n",
      "10177it [04:23, 52.94it/s]\u001b[A\n",
      "10209it [04:24, 53.10it/s]\u001b[A\n",
      "10241it [04:25, 52.92it/s]\u001b[A\n",
      "10273it [04:25, 52.57it/s]\u001b[A\n",
      "10305it [04:26, 52.57it/s]\u001b[A\n",
      "10337it [04:26, 52.83it/s]\u001b[A\n",
      "10369it [04:27, 52.72it/s]\u001b[A\n",
      "10401it [04:28, 52.95it/s]\u001b[A\n",
      "10433it [04:28, 52.79it/s]\u001b[A\n",
      "10465it [04:29, 52.60it/s]\u001b[A\n",
      "10497it [04:30, 52.49it/s]\u001b[A\n",
      "10529it [04:30, 52.75it/s]\u001b[A\n",
      "10561it [04:31, 52.93it/s]\u001b[A\n",
      "10593it [04:31, 52.84it/s]\u001b[A\n",
      "10625it [04:32, 52.80it/s]\u001b[A\n",
      "10657it [04:33, 52.97it/s]\u001b[A\n",
      "10689it [04:33, 52.82it/s]\u001b[A\n",
      "10721it [04:34, 53.02it/s]\u001b[A\n",
      "10753it [04:34, 52.83it/s]\u001b[A\n",
      "10785it [04:35, 52.61it/s]\u001b[A\n",
      "10817it [04:36, 52.76it/s]\u001b[A\n",
      "10849it [04:36, 52.79it/s]\u001b[A\n",
      "10881it [04:37, 52.67it/s]\u001b[A\n",
      "10913it [04:37, 52.90it/s]\u001b[A\n",
      "10945it [04:38, 52.60it/s]\u001b[A\n",
      "10977it [04:39, 52.59it/s]\u001b[A\n",
      "11009it [04:39, 52.82it/s]\u001b[A\n",
      "11041it [04:40, 52.65it/s]\u001b[A\n",
      "11073it [04:40, 52.64it/s]\u001b[A\n",
      "11105it [04:41, 52.83it/s]\u001b[A\n",
      "11137it [04:42, 52.73it/s]\u001b[A\n",
      "11169it [04:42, 52.97it/s]\u001b[A\n",
      "11201it [04:43, 52.70it/s]\u001b[A\n",
      "11233it [04:43, 52.69it/s]\u001b[A\n",
      "11265it [04:44, 52.48it/s]\u001b[A\n",
      "11297it [04:45, 52.39it/s]\u001b[A\n",
      "11329it [04:45, 52.70it/s]\u001b[A\n",
      "11361it [04:46, 52.67it/s]\u001b[A\n",
      "11393it [04:47, 52.88it/s]\u001b[A\n",
      "11425it [04:47, 52.73it/s]\u001b[A\n",
      "11457it [04:48, 53.01it/s]\u001b[A\n",
      "11489it [04:48, 53.07it/s]\u001b[A\n",
      "11521it [04:49, 53.11it/s]\u001b[A\n",
      "11553it [04:50, 52.80it/s]\u001b[A\n",
      "11585it [04:50, 53.00it/s]\u001b[A\n",
      "11617it [04:51, 52.87it/s]\u001b[A\n",
      "11649it [04:51, 53.04it/s]\u001b[A\n",
      "11681it [04:52, 52.81it/s]\u001b[A\n",
      "11713it [04:53, 52.57it/s]\u001b[A\n",
      "11745it [04:53, 52.54it/s]\u001b[A\n",
      "11777it [04:54, 52.77it/s]\u001b[A\n",
      "11809it [04:54, 52.71it/s]\u001b[A\n",
      "11841it [04:55, 52.59it/s]\u001b[A\n",
      "11873it [04:56, 52.78it/s]\u001b[A\n",
      "11905it [04:56, 52.72it/s]\u001b[A\n",
      "11937it [04:57, 52.58it/s]\u001b[A\n",
      "11969it [04:57, 52.84it/s]\u001b[A\n",
      "12001it [04:58, 52.78it/s]\u001b[A\n",
      "12033it [04:59, 52.42it/s]\u001b[A\n",
      "12065it [04:59, 52.33it/s]\u001b[A\n",
      "12097it [05:00, 52.35it/s]\u001b[A\n",
      "12129it [05:00, 52.67it/s]\u001b[A\n",
      "12161it [05:01, 52.61it/s]\u001b[A\n",
      "12193it [05:02, 52.52it/s]\u001b[A\n",
      "12225it [05:02, 52.80it/s]\u001b[A\n",
      "12257it [05:03, 52.57it/s]\u001b[A\n",
      "12289it [05:04, 52.62it/s]\u001b[A\n",
      "12321it [05:04, 52.85it/s]\u001b[A\n",
      "12353it [05:05, 52.78it/s]\u001b[A\n",
      "12385it [05:05, 52.59it/s]\u001b[A\n",
      "12417it [05:06, 52.84it/s]\u001b[A\n",
      "12449it [05:07, 53.09it/s]\u001b[A\n",
      "12481it [05:07, 52.81it/s]\u001b[A\n",
      "12513it [05:08, 52.71it/s]\u001b[A\n",
      "12545it [05:08, 52.51it/s]\u001b[A\n",
      "12577it [05:09, 52.63it/s]\u001b[A\n",
      "12609it [05:10, 52.82it/s]\u001b[A\n",
      "12641it [05:10, 52.99it/s]\u001b[A\n",
      "12673it [05:11, 52.86it/s]\u001b[A\n",
      "12705it [05:11, 53.01it/s]\u001b[A\n",
      "12737it [05:12, 52.88it/s]\u001b[A\n",
      "12769it [05:13, 52.53it/s]\u001b[A\n",
      "12801it [05:13, 52.57it/s]\u001b[A\n",
      "12833it [05:14, 52.76it/s]\u001b[A\n",
      "12865it [05:14, 52.65it/s]\u001b[A\n",
      "12897it [05:15, 52.44it/s]\u001b[A\n",
      "12929it [05:16, 52.51it/s]\u001b[A\n",
      "12961it [05:16, 52.76it/s]\u001b[A\n",
      "12993it [05:17, 52.70it/s]\u001b[A\n",
      "13025it [05:17, 52.92it/s]\u001b[A\n",
      "13057it [05:18, 52.79it/s]\u001b[A\n",
      "13089it [05:19, 52.99it/s]\u001b[A\n",
      "13121it [05:19, 52.80it/s]\u001b[A\n",
      "13153it [05:20, 52.65it/s]\u001b[A\n",
      "13185it [05:20, 52.88it/s]\u001b[A\n",
      "13217it [05:21, 53.00it/s]\u001b[A\n",
      "13249it [05:22, 52.84it/s]\u001b[A\n",
      "13281it [05:22, 52.83it/s]\u001b[A\n",
      "13313it [05:23, 52.87it/s]\u001b[A\n",
      "13345it [05:24, 52.90it/s]\u001b[A\n",
      "13377it [05:24, 53.02it/s]\u001b[A\n",
      "13409it [05:25, 52.87it/s]\u001b[A\n",
      "13441it [05:25, 53.04it/s]\u001b[A\n",
      "13473it [05:26, 52.80it/s]\u001b[A\n",
      "13505it [05:27, 52.70it/s]\u001b[A\n",
      "13537it [05:27, 52.48it/s]\u001b[A\n",
      "13569it [05:28, 52.37it/s]\u001b[A\n",
      "13601it [05:28, 52.39it/s]\u001b[A\n",
      "13633it [05:29, 52.21it/s]\u001b[A\n",
      "13665it [05:30, 52.22it/s]\u001b[A\n",
      "13697it [05:30, 52.56it/s]\u001b[A\n",
      "13729it [05:31, 52.49it/s]\u001b[A\n",
      "13761it [05:31, 52.34it/s]\u001b[A\n",
      "13793it [05:32, 52.29it/s]\u001b[A\n",
      "13825it [05:33, 52.26it/s]\u001b[A\n",
      "13857it [05:33, 52.32it/s]\u001b[A\n",
      "13889it [05:34, 52.15it/s]\u001b[A\n",
      "13921it [05:34, 52.31it/s]\u001b[A\n",
      "13953it [05:35, 52.62it/s]\u001b[A\n",
      "13985it [05:36, 52.61it/s]\u001b[A\n",
      "14017it [05:36, 52.48it/s]\u001b[A\n",
      "14049it [05:37, 52.73it/s]\u001b[A\n",
      "14081it [05:38, 52.57it/s]\u001b[A\n",
      "14113it [05:38, 52.57it/s]\u001b[A\n",
      "14145it [05:39, 52.81it/s]\u001b[A\n",
      "14177it [05:39, 52.58it/s]\u001b[A\n",
      "14209it [05:40, 52.53it/s]\u001b[A\n",
      "14241it [05:41, 52.47it/s]\u001b[A\n",
      "14273it [05:41, 52.64it/s]\u001b[A\n",
      "14305it [05:42, 52.67it/s]\u001b[A\n",
      "14337it [05:42, 52.88it/s]\u001b[A\n",
      "14369it [05:43, 52.77it/s]\u001b[A\n",
      "14401it [05:44, 52.98it/s]\u001b[A\n",
      "14433it [05:44, 52.86it/s]\u001b[A\n",
      "14465it [05:45, 53.00it/s]\u001b[A\n",
      "14497it [05:45, 52.76it/s]\u001b[A\n",
      "14529it [05:46, 52.56it/s]\u001b[A\n",
      "14561it [05:47, 52.53it/s]\u001b[A\n",
      "14593it [05:47, 52.81it/s]\u001b[A\n",
      "14625it [05:48, 52.68it/s]\u001b[A\n",
      "14657it [05:48, 52.42it/s]\u001b[A\n",
      "14689it [05:49, 52.48it/s]\u001b[A\n",
      "14721it [05:50, 52.73it/s]\u001b[A\n",
      "14753it [05:50, 52.61it/s]\u001b[A\n",
      "14785it [05:51, 52.74it/s]\u001b[A\n",
      "14817it [05:51, 52.71it/s]\u001b[A\n",
      "14849it [05:52, 52.61it/s]\u001b[A\n",
      "14881it [05:53, 52.43it/s]\u001b[A\n",
      "14913it [05:53, 52.40it/s]\u001b[A\n",
      "14945it [05:54, 52.42it/s]\u001b[A\n",
      "14977it [05:55, 52.55it/s]\u001b[A\n",
      "15009it [05:55, 52.56it/s]\u001b[A\n",
      "15041it [05:56, 52.33it/s]\u001b[A\n",
      "15073it [05:56, 52.41it/s]\u001b[A\n",
      "15105it [05:57, 52.17it/s]\u001b[A\n",
      "15137it [05:58, 52.18it/s]\u001b[A\n",
      "15169it [05:58, 52.12it/s]\u001b[A\n",
      "15201it [05:59, 52.28it/s]\u001b[A\n",
      "15233it [05:59, 52.60it/s]\u001b[A\n",
      "15265it [06:00, 52.65it/s]\u001b[A\n",
      "15297it [06:01, 52.86it/s]\u001b[A\n",
      "15329it [06:01, 52.79it/s]\u001b[A\n",
      "15361it [06:02, 52.94it/s]\u001b[A\n",
      "15393it [06:02, 52.75it/s]\u001b[A\n",
      "15425it [06:03, 52.60it/s]\u001b[A\n",
      "15457it [06:04, 52.36it/s]\u001b[A\n",
      "15489it [06:04, 52.36it/s]\u001b[A\n",
      "15521it [06:05, 52.37it/s]\u001b[A\n",
      "15553it [06:06, 52.33it/s]\u001b[A\n",
      "15585it [06:06, 52.62it/s]\u001b[A\n",
      "15617it [06:07, 52.81it/s]\u001b[A\n",
      "15649it [06:07, 52.78it/s]\u001b[A\n",
      "15681it [06:08, 52.70it/s]\u001b[A\n",
      "15713it [06:09, 52.92it/s]\u001b[A\n",
      "15745it [06:09, 52.70it/s]\u001b[A\n",
      "15777it [06:10, 52.63it/s]\u001b[A\n",
      "15809it [06:10, 52.85it/s]\u001b[A\n",
      "15841it [06:11, 52.77it/s]\u001b[A\n",
      "15873it [06:12, 52.98it/s]\u001b[A\n",
      "15905it [06:12, 52.77it/s]\u001b[A\n",
      "15937it [06:13, 52.84it/s]\u001b[A\n",
      "15969it [06:13, 52.92it/s]\u001b[A\n",
      "16001it [06:14, 52.67it/s]\u001b[A\n",
      "16033it [06:15, 52.48it/s]\u001b[A\n",
      "16065it [06:15, 52.42it/s]\u001b[A\n",
      "16097it [06:16, 52.31it/s]\u001b[A\n",
      "16129it [06:16, 52.59it/s]\u001b[A\n",
      "16161it [06:17, 52.45it/s]\u001b[A\n",
      "16193it [06:18, 52.34it/s]\u001b[A\n",
      "16225it [06:18, 52.28it/s]\u001b[A\n",
      "16257it [06:19, 52.43it/s]\u001b[A\n",
      "16289it [06:19, 52.72it/s]\u001b[A\n",
      "16321it [06:20, 52.66it/s]\u001b[A\n",
      "16353it [06:21, 52.89it/s]\u001b[A\n",
      "16385it [06:21, 52.72it/s]\u001b[A\n",
      "16417it [06:22, 52.62it/s]\u001b[A\n",
      "16449it [06:23, 52.40it/s]\u001b[A\n",
      "16481it [06:23, 52.72it/s]\u001b[A\n",
      "16513it [06:24, 52.58it/s]\u001b[A\n",
      "16545it [06:24, 52.47it/s]\u001b[A\n",
      "16577it [06:25, 52.38it/s]\u001b[A\n",
      "16609it [06:26, 52.28it/s]\u001b[A\n",
      "16641it [06:26, 52.26it/s]\u001b[A\n",
      "16673it [06:27, 52.26it/s]\u001b[A\n",
      "16705it [06:27, 52.28it/s]\u001b[A\n",
      "16737it [06:28, 52.14it/s]\u001b[A\n",
      "16769it [06:29, 52.18it/s]\u001b[A\n",
      "16801it [06:29, 52.23it/s]\u001b[A\n",
      "16833it [06:30, 52.56it/s]\u001b[A\n",
      "16865it [06:30, 52.50it/s]\u001b[A\n",
      "16897it [06:31, 52.35it/s]\u001b[A\n",
      "16929it [06:32, 52.23it/s]\u001b[A\n",
      "16961it [06:32, 52.35it/s]\u001b[A\n",
      "16993it [06:33, 52.56it/s]\u001b[A\n",
      "17025it [06:34, 52.61it/s]\u001b[A\n",
      "17057it [06:34, 52.43it/s]\u001b[A\n",
      "17089it [06:35, 52.34it/s]\u001b[A\n",
      "17121it [06:35, 52.16it/s]\u001b[A\n",
      "17153it [06:36, 52.28it/s]\u001b[A\n",
      "17185it [06:37, 52.23it/s]\u001b[A\n",
      "17217it [06:37, 52.51it/s]\u001b[A\n",
      "17249it [06:38, 52.46it/s]\u001b[A\n",
      "17281it [06:38, 52.56it/s]\u001b[A\n",
      "17313it [06:39, 52.64it/s]\u001b[A\n",
      "17345it [06:40, 52.60it/s]\u001b[A\n",
      "17377it [06:40, 52.40it/s]\u001b[A\n",
      "17409it [06:41, 52.31it/s]\u001b[A\n",
      "17441it [06:41, 52.46it/s]\u001b[A\n",
      "17473it [06:42, 52.74it/s]\u001b[A\n",
      "17505it [06:43, 52.66it/s]\u001b[A\n",
      "17537it [06:43, 52.90it/s]\u001b[A\n",
      "17569it [06:44, 52.72it/s]\u001b[A\n",
      "17601it [06:45, 52.49it/s]\u001b[A\n",
      "17633it [06:45, 52.43it/s]\u001b[A\n",
      "17665it [06:46, 52.34it/s]\u001b[A\n",
      "17697it [06:46, 52.21it/s]\u001b[A\n",
      "17729it [06:47, 52.31it/s]\u001b[A\n",
      "17761it [06:48, 52.06it/s]\u001b[A\n",
      "17793it [06:48, 52.25it/s]\u001b[A\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "17825it [06:49, 52.50it/s]\u001b[A\n",
      "17857it [06:49, 52.63it/s]\u001b[A\n",
      "17889it [06:50, 52.49it/s]\u001b[A\n",
      "17921it [06:51, 52.38it/s]\u001b[A\n",
      "17953it [06:51, 52.65it/s]\u001b[A\n",
      "17985it [06:52, 52.61it/s]\u001b[A\n",
      "18017it [06:52, 52.87it/s]\u001b[A\n",
      "18049it [06:53, 52.70it/s]\u001b[A\n",
      "18081it [06:54, 52.60it/s]\u001b[A\n",
      "18113it [06:54, 52.48it/s]\u001b[A\n",
      "18145it [06:55, 52.29it/s]\u001b[A\n",
      "18177it [06:56, 52.12it/s]\u001b[A\n",
      "18209it [06:56, 52.14it/s]\u001b[A\n",
      "18241it [06:57, 52.25it/s]\u001b[A\n",
      "18273it [06:57, 52.54it/s]\u001b[A\n",
      "18305it [06:58, 52.47it/s]\u001b[A\n",
      "18337it [06:59, 52.49it/s]\u001b[A\n",
      "18369it [06:59, 52.77it/s]\u001b[A\n",
      "18401it [07:00, 52.64it/s]\u001b[A\n",
      "18433it [07:00, 52.37it/s]\u001b[A\n",
      "18465it [07:01, 52.37it/s]\u001b[A\n",
      "18497it [07:02, 52.28it/s]\u001b[A\n",
      "18529it [07:02, 52.25it/s]\u001b[A\n",
      "18561it [07:03, 52.32it/s]\u001b[A\n",
      "18593it [07:03, 52.22it/s]\u001b[A\n",
      "18625it [07:04, 52.57it/s]\u001b[A\n",
      "18657it [07:05, 52.56it/s]\u001b[A\n",
      "18689it [07:05, 52.80it/s]\u001b[A\n",
      "18721it [07:06, 52.66it/s]\u001b[A\n",
      "18753it [07:06, 52.47it/s]\u001b[A\n",
      "18785it [07:07, 52.65it/s]\u001b[A\n",
      "18817it [07:08, 52.59it/s]\u001b[A\n",
      "18849it [07:08, 52.65it/s]\u001b[A\n",
      "18881it [07:09, 52.80it/s]\u001b[A\n",
      "18913it [07:10, 52.66it/s]\u001b[A\n",
      "18945it [07:10, 52.63it/s]\u001b[A\n",
      "18977it [07:11, 52.36it/s]\u001b[A\n",
      "19009it [07:11, 52.30it/s]\u001b[A\n",
      "19041it [07:12, 52.25it/s]\u001b[A\n",
      "19073it [07:13, 52.30it/s]\u001b[A\n",
      "19105it [07:13, 52.22it/s]\u001b[A\n",
      "19137it [07:14, 52.18it/s]\u001b[A\n",
      "19169it [07:14, 52.02it/s]\u001b[A\n",
      "19201it [07:15, 52.05it/s]\u001b[A\n",
      "19233it [07:16, 52.09it/s]\u001b[A\n",
      "19265it [07:16, 52.15it/s]\u001b[A\n",
      "19297it [07:17, 52.12it/s]\u001b[A\n",
      "19329it [07:17, 52.16it/s]\u001b[A\n",
      "19361it [07:18, 52.17it/s]\u001b[A\n",
      "19393it [07:19, 52.05it/s]\u001b[A\n",
      "19425it [07:19, 52.23it/s]\u001b[A\n",
      "19457it [07:20, 52.52it/s]\u001b[A\n",
      "19489it [07:21, 52.49it/s]\u001b[A\n",
      "19521it [07:21, 52.32it/s]\u001b[A\n",
      "19553it [07:22, 52.33it/s]\u001b[A\n",
      "19585it [07:22, 52.26it/s]\u001b[A\n",
      "19617it [07:23, 52.31it/s]\u001b[A\n",
      "19649it [07:24, 52.43it/s]\u001b[A\n",
      "19681it [07:24, 52.47it/s]\u001b[A\n",
      "19713it [07:25, 52.41it/s]\u001b[A\n",
      "19745it [07:25, 52.34it/s]\u001b[A\n",
      "19777it [07:26, 52.39it/s]\u001b[A\n",
      "19809it [07:27, 52.63it/s]\u001b[A\n",
      "19841it [07:27, 52.52it/s]\u001b[A\n",
      "19873it [07:28, 52.37it/s]\u001b[A\n",
      "19905it [07:28, 52.44it/s]\u001b[A\n",
      "19937it [07:29, 52.44it/s]\u001b[A\n",
      "19969it [07:30, 52.63it/s]\u001b[A\n",
      "20001it [07:30, 52.64it/s]\u001b[A\n",
      "20033it [07:31, 52.86it/s]\u001b[A\n",
      "20065it [07:32, 52.57it/s]\u001b[A\n",
      "20097it [07:32, 52.38it/s]\u001b[A\n",
      "20129it [07:33, 52.44it/s]\u001b[A\n",
      "20161it [07:33, 52.29it/s]\u001b[A\n",
      "20193it [07:34, 52.41it/s]\u001b[A\n",
      "20225it [07:35, 52.54it/s]\u001b[A\n",
      "20257it [07:35, 52.70it/s]\u001b[A\n",
      "20289it [07:36, 52.79it/s]\u001b[A\n",
      "20321it [07:36, 52.69it/s]\u001b[A\n",
      "20353it [07:37, 52.50it/s]\u001b[A\n",
      "20385it [07:38, 52.39it/s]\u001b[A\n",
      "20417it [07:38, 52.35it/s]\u001b[A\n",
      "20449it [07:39, 52.27it/s]\u001b[A\n",
      "20481it [07:39, 52.26it/s]\u001b[A\n",
      "20513it [07:40, 52.14it/s]\u001b[A\n",
      "20545it [07:41, 52.22it/s]\u001b[A\n",
      "20577it [07:41, 52.24it/s]\u001b[A\n",
      "20609it [07:42, 52.13it/s]\u001b[A\n",
      "20641it [07:43, 52.12it/s]\u001b[A\n",
      "20673it [07:43, 52.06it/s]\u001b[A\n",
      "20705it [07:44, 52.24it/s]\u001b[A\n",
      "20737it [07:44, 52.06it/s]\u001b[A\n",
      "20769it [07:45, 52.08it/s]\u001b[A\n",
      "20801it [07:46, 52.25it/s]\u001b[A\n",
      "20833it [07:46, 52.07it/s]\u001b[A\n",
      "20865it [07:47, 52.06it/s]\u001b[A\n",
      "20897it [07:47, 52.17it/s]\u001b[A\n",
      "20929it [07:48, 52.06it/s]\u001b[A\n",
      "20961it [07:49, 52.10it/s]\u001b[A\n",
      "20993it [07:49, 52.10it/s]\u001b[A\n",
      "21025it [07:50, 52.13it/s]\u001b[A\n",
      "21057it [07:51, 52.16it/s]\u001b[A\n",
      "21089it [07:51, 52.24it/s]\u001b[A\n",
      "21121it [07:52, 52.60it/s]\u001b[A\n",
      "21153it [07:52, 52.45it/s]\u001b[A\n",
      "21185it [07:53, 52.34it/s]\u001b[A\n",
      "21217it [07:54, 52.27it/s]\u001b[A\n",
      "21249it [07:54, 52.28it/s]\u001b[A\n",
      "21281it [07:55, 52.20it/s]\u001b[A\n",
      "21313it [07:55, 52.11it/s]\u001b[A\n",
      "21345it [07:56, 52.17it/s]\u001b[A\n",
      "21377it [07:57, 52.22it/s]\u001b[A\n",
      "21409it [07:57, 52.40it/s]\u001b[A\n",
      "21441it [07:58, 52.49it/s]\u001b[A\n",
      "21473it [07:58, 52.33it/s]\u001b[A\n",
      "21505it [07:59, 52.30it/s]\u001b[A\n",
      "21537it [08:00, 52.25it/s]\u001b[A\n",
      "21569it [08:00, 52.23it/s]\u001b[A\n",
      "21601it [08:01, 52.16it/s]\u001b[A\n",
      "21633it [08:02, 52.14it/s]\u001b[A\n",
      "21665it [08:02, 52.14it/s]\u001b[A\n",
      "21697it [08:03, 52.14it/s]\u001b[A\n",
      "21729it [08:03, 52.15it/s]\u001b[A\n",
      "21761it [08:04, 52.03it/s]\u001b[A\n",
      "21793it [08:05, 52.13it/s]\u001b[A\n",
      "21825it [08:05, 52.12it/s]\u001b[A\n",
      "21857it [08:06, 52.10it/s]\u001b[A\n",
      "21889it [08:06, 52.17it/s]\u001b[A\n",
      "21921it [08:07, 52.17it/s]\u001b[A\n",
      "21953it [08:08, 52.15it/s]\u001b[A\n",
      "21985it [08:08, 52.15it/s]\u001b[A\n",
      "22017it [08:09, 52.06it/s]\u001b[A\n",
      "22049it [08:10, 52.14it/s]\u001b[A\n",
      "22081it [08:10, 52.15it/s]\u001b[A\n",
      "22113it [08:11, 52.09it/s]\u001b[A\n",
      "22145it [08:11, 52.15it/s]\u001b[A\n",
      "22177it [08:12, 52.04it/s]\u001b[A\n",
      "22209it [08:13, 52.11it/s]\u001b[A\n",
      "22241it [08:13, 52.22it/s]\u001b[A\n",
      "22273it [08:14, 52.55it/s]\u001b[A\n",
      "22305it [08:14, 52.60it/s]\u001b[A\n",
      "22337it [08:15, 52.78it/s]\u001b[A\n",
      "22369it [08:16, 52.59it/s]\u001b[A\n",
      "22401it [08:16, 52.49it/s]\u001b[A\n",
      "22433it [08:17, 52.50it/s]\u001b[A\n",
      "22465it [08:17, 52.33it/s]\u001b[A\n",
      "22497it [08:18, 52.29it/s]\u001b[A\n",
      "22529it [08:19, 52.19it/s]\u001b[A\n",
      "22561it [08:19, 52.13it/s]\u001b[A\n",
      "22593it [08:20, 52.06it/s]\u001b[A\n",
      "22625it [08:21, 52.14it/s]\u001b[A\n",
      "22657it [08:21, 52.13it/s]\u001b[A\n",
      "22689it [08:22, 52.16it/s]\u001b[A\n",
      "22721it [08:22, 52.22it/s]\u001b[A\n",
      "22753it [08:23, 52.10it/s]\u001b[A\n",
      "22785it [08:24, 52.14it/s]\u001b[A\n",
      "22817it [08:24, 52.12it/s]\u001b[A\n",
      "22849it [08:25, 52.03it/s]\u001b[A\n",
      "22881it [08:25, 52.30it/s]\u001b[A\n",
      "22913it [08:26, 52.20it/s]\u001b[A\n",
      "22945it [08:27, 52.10it/s]\u001b[A\n",
      "22977it [08:27, 52.11it/s]\u001b[A\n",
      "23009it [08:28, 52.08it/s]\u001b[A\n",
      "23041it [08:28, 52.09it/s]\u001b[A\n",
      "23073it [08:29, 52.26it/s]\u001b[A\n",
      "23105it [08:30, 52.22it/s]\u001b[A\n",
      "23137it [08:30, 52.20it/s]\u001b[A\n",
      "23169it [08:31, 52.35it/s]\u001b[A\n",
      "23201it [08:32, 52.43it/s]\u001b[A\n",
      "23233it [08:32, 52.35it/s]\u001b[A\n",
      "23265it [08:33, 52.29it/s]\u001b[A\n",
      "23297it [08:33, 52.23it/s]\u001b[A\n",
      "23329it [08:34, 52.16it/s]\u001b[A\n",
      "23361it [08:35, 52.18it/s]\u001b[A\n",
      "23393it [08:35, 52.08it/s]\u001b[A\n",
      "23425it [08:36, 52.25it/s]\u001b[A\n",
      "23457it [08:36, 52.16it/s]\u001b[A\n",
      "23489it [08:37, 52.15it/s]\u001b[A\n",
      "23521it [08:38, 52.12it/s]\u001b[A\n",
      "23553it [08:38, 52.22it/s]\u001b[A\n",
      "23585it [08:39, 52.15it/s]\u001b[A\n",
      "23617it [08:40, 52.14it/s]\u001b[A\n",
      "23649it [08:40, 52.11it/s]\u001b[A\n",
      "23681it [08:41, 52.16it/s]\u001b[A\n",
      "23713it [08:41, 52.12it/s]\u001b[A\n",
      "23745it [08:42, 52.07it/s]\u001b[A\n",
      "23777it [08:43, 52.15it/s]\u001b[A\n",
      "23809it [08:43, 52.16it/s]\u001b[A\n",
      "23841it [08:44, 52.20it/s]\u001b[A\n",
      "23873it [08:44, 52.10it/s]\u001b[A\n",
      "23905it [08:45, 52.19it/s]\u001b[A\n",
      "23937it [08:46, 52.16it/s]\u001b[A\n",
      "23969it [08:46, 52.10it/s]\u001b[A\n",
      "24001it [08:47, 52.14it/s]\u001b[A\n",
      "24033it [08:48, 52.09it/s]\u001b[A\n",
      "24065it [08:48, 52.13it/s]\u001b[A\n",
      "24097it [08:49, 52.15it/s]\u001b[A\n",
      "24129it [08:49, 52.18it/s]\u001b[A\n",
      "24161it [08:50, 52.10it/s]\u001b[A\n",
      "24193it [08:51, 52.13it/s]\u001b[A\n",
      "24225it [08:51, 51.99it/s]\u001b[A\n",
      "24257it [08:52, 52.10it/s]\u001b[A\n",
      "24289it [08:52, 52.14it/s]\u001b[A\n",
      "24321it [08:53, 52.13it/s]\u001b[A\n",
      "24353it [08:54, 52.10it/s]\u001b[A\n",
      "24385it [08:54, 52.07it/s]\u001b[A\n",
      "24417it [08:55, 52.04it/s]\u001b[A\n",
      "24449it [08:55, 52.12it/s]\u001b[A\n",
      "24481it [08:56, 52.06it/s]\u001b[A\n",
      "24513it [08:57, 52.11it/s]\u001b[A\n",
      "24545it [08:57, 52.07it/s]\u001b[A\n",
      "24577it [08:58, 52.12it/s]\u001b[A\n",
      "24609it [08:59, 52.14it/s]\u001b[A\n",
      "24641it [08:59, 52.10it/s]\u001b[A\n",
      "24673it [09:00, 52.17it/s]\u001b[A\n",
      "24705it [09:00, 52.11it/s]\u001b[A\n",
      "24737it [09:01, 52.13it/s]\u001b[A\n",
      "24769it [09:02, 52.17it/s]\u001b[A\n",
      "24801it [09:02, 52.09it/s]\u001b[A\n",
      "24833it [09:03, 52.18it/s]\u001b[A\n",
      "24865it [09:03, 52.10it/s]\u001b[A\n",
      "24897it [09:04, 52.12it/s]\u001b[A\n",
      "24929it [09:05, 52.16it/s]\u001b[A\n",
      "24961it [09:05, 52.14it/s]\u001b[A\n",
      "24993it [09:06, 52.10it/s]\u001b[A\n",
      "25025it [09:07, 52.04it/s]\u001b[A\n",
      "25057it [09:07, 52.05it/s]\u001b[A\n",
      "25089it [09:08, 52.08it/s]\u001b[A\n",
      "25121it [09:08, 52.18it/s]\u001b[A\n",
      "25153it [09:09, 52.02it/s]\u001b[A\n",
      "25185it [09:10, 52.25it/s]\u001b[A\n",
      "25217it [09:10, 52.49it/s]\u001b[A\n",
      "25249it [09:11, 52.57it/s]\u001b[A\n",
      "25281it [09:11, 52.24it/s]\u001b[A\n",
      "25313it [09:12, 52.24it/s]\u001b[A\n",
      "25345it [09:13, 52.24it/s]\u001b[A\n",
      "25377it [09:13, 52.21it/s]\u001b[A\n",
      "25409it [09:14, 52.14it/s]\u001b[A\n",
      "25441it [09:15, 52.14it/s]\u001b[A\n",
      "25473it [09:15, 52.25it/s]\u001b[A\n",
      "25505it [09:16, 52.20it/s]\u001b[A\n",
      "25537it [09:16, 52.02it/s]\u001b[A\n",
      "25569it [09:17, 52.13it/s]\u001b[A\n",
      "25601it [09:18, 52.11it/s]\u001b[A\n",
      "25633it [09:18, 52.12it/s]\u001b[A\n",
      "25665it [09:19, 52.10it/s]\u001b[A\n",
      "25697it [09:19, 52.16it/s]\u001b[A\n",
      "25729it [09:20, 52.20it/s]\u001b[A\n",
      "25761it [09:21, 52.14it/s]\u001b[A\n",
      "25793it [09:21, 52.13it/s]\u001b[A\n",
      "25825it [09:22, 52.15it/s]\u001b[A\n",
      "25857it [09:22, 52.15it/s]\u001b[A\n",
      "25889it [09:23, 52.12it/s]\u001b[A\n",
      "25921it [09:24, 52.08it/s]\u001b[A\n",
      "25953it [09:24, 52.10it/s]\u001b[A\n",
      "25985it [09:25, 52.08it/s]\u001b[A\n",
      "26017it [09:26, 52.06it/s]\u001b[A\n",
      "26049it [09:26, 52.08it/s]\u001b[A\n",
      "26081it [09:27, 52.21it/s]\u001b[A\n",
      "26113it [09:27, 52.07it/s]\u001b[A\n",
      "26145it [09:28, 52.22it/s]\u001b[A\n",
      "26177it [09:29, 52.12it/s]\u001b[A\n",
      "26209it [09:29, 52.08it/s]\u001b[A\n",
      "26241it [09:30, 52.06it/s]\u001b[A\n",
      "26273it [09:30, 52.14it/s]\u001b[A\n",
      "26305it [09:31, 52.06it/s]\u001b[A\n",
      "26337it [09:32, 52.10it/s]\u001b[A\n",
      "26369it [09:32, 52.10it/s]\u001b[A\n",
      "26401it [09:33, 52.18it/s]\u001b[A\n",
      "26433it [09:34, 52.13it/s]\u001b[A\n",
      "26465it [09:34, 52.11it/s]\u001b[A\n",
      "26497it [09:35, 52.27it/s]\u001b[A\n",
      "26529it [09:35, 52.11it/s]\u001b[A\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "26561it [09:36, 52.14it/s]\u001b[A\n",
      "26593it [09:37, 52.17it/s]\u001b[A\n",
      "26625it [09:37, 52.18it/s]\u001b[A\n",
      "26657it [09:38, 52.02it/s]\u001b[A\n",
      "26689it [09:38, 52.05it/s]\u001b[A\n",
      "26721it [09:39, 52.07it/s]\u001b[A\n",
      "26753it [09:40, 52.05it/s]\u001b[A\n",
      "26785it [09:40, 52.05it/s]\u001b[A\n",
      "26817it [09:41, 52.13it/s]\u001b[A\n",
      "26849it [09:42, 52.20it/s]\u001b[A\n",
      "26881it [09:42, 52.12it/s]\u001b[A\n",
      "26913it [09:43, 52.07it/s]\u001b[A\n",
      "26945it [09:43, 52.13it/s]\u001b[A\n",
      "26977it [09:44, 52.10it/s]\u001b[A\n",
      "27009it [09:45, 52.11it/s]\u001b[A\n",
      "27041it [09:45, 52.13it/s]\u001b[A\n",
      "27073it [09:46, 52.06it/s]\u001b[A\n",
      "27105it [09:46, 52.21it/s]\u001b[A\n",
      "27137it [09:47, 52.02it/s]\u001b[A\n",
      "27169it [09:48, 52.12it/s]\u001b[A\n",
      "27201it [09:48, 52.06it/s]\u001b[A\n",
      "27233it [09:49, 52.09it/s]\u001b[A\n",
      "27265it [09:50, 52.11it/s]\u001b[A\n",
      "27297it [09:50, 52.13it/s]\u001b[A\n",
      "27329it [09:51, 52.11it/s]\u001b[A\n",
      "27361it [09:51, 52.16it/s]\u001b[A\n",
      "27393it [09:52, 52.13it/s]\u001b[A\n",
      "27425it [09:53, 52.07it/s]\u001b[A\n",
      "27457it [09:53, 52.09it/s]\u001b[A\n",
      "27489it [09:54, 52.03it/s]\u001b[A\n",
      "27521it [09:54, 52.20it/s]\u001b[A\n",
      "27553it [09:55, 52.31it/s]\u001b[A\n",
      "27585it [09:56, 52.17it/s]\u001b[A\n",
      "27617it [09:56, 52.04it/s]\u001b[A\n",
      "27649it [09:57, 52.25it/s]\u001b[A\n",
      "27681it [09:57, 52.07it/s]\u001b[A\n",
      "27713it [09:58, 52.23it/s]\u001b[A\n",
      "27745it [09:59, 52.06it/s]\u001b[A\n",
      "27777it [09:59, 52.10it/s]\u001b[A\n",
      "27809it [10:00, 52.09it/s]\u001b[A\n",
      "27841it [10:01, 52.11it/s]\u001b[A\n",
      "27873it [10:01, 52.08it/s]\u001b[A\n",
      "27905it [10:02, 52.06it/s]\u001b[A\n",
      "27937it [10:02, 52.00it/s]\u001b[A\n",
      "27969it [10:03, 52.15it/s]\u001b[A\n",
      "28001it [10:04, 52.03it/s]\u001b[A\n",
      "28033it [10:04, 52.33it/s]\u001b[A\n",
      "28065it [10:05, 52.16it/s]\u001b[A\n",
      "28097it [10:05, 52.09it/s]\u001b[A\n",
      "28129it [10:06, 52.08it/s]\u001b[A\n",
      "28161it [10:07, 52.10it/s]\u001b[A\n",
      "28193it [10:07, 52.14it/s]\u001b[A\n",
      "28225it [10:08, 52.15it/s]\u001b[A\n",
      "28257it [10:09, 52.04it/s]\u001b[A\n",
      "28289it [10:09, 52.18it/s]\u001b[A\n",
      "28321it [10:10, 52.13it/s]\u001b[A\n",
      "28353it [10:10, 52.08it/s]\u001b[A\n",
      "28385it [10:11, 52.15it/s]\u001b[A\n",
      "28417it [10:12, 52.17it/s]\u001b[A\n",
      "28449it [10:12, 52.21it/s]\u001b[A\n",
      "28481it [10:13, 52.15it/s]\u001b[A\n",
      "28513it [10:13, 52.03it/s]\u001b[A\n",
      "28545it [10:14, 52.15it/s]\u001b[A\n",
      "28577it [10:15, 52.11it/s]\u001b[A\n",
      "28609it [10:15, 52.31it/s]\u001b[A\n",
      "28641it [10:16, 52.62it/s]\u001b[A\n",
      "28673it [10:16, 52.46it/s]\u001b[A\n",
      "28705it [10:17, 52.37it/s]\u001b[A\n",
      "28737it [10:18, 52.31it/s]\u001b[A\n",
      "28769it [10:18, 52.24it/s]\u001b[A\n",
      "28801it [10:19, 52.15it/s]\u001b[A\n",
      "28833it [10:20, 52.17it/s]\u001b[A\n",
      "28865it [10:20, 52.19it/s]\u001b[A\n",
      "28897it [10:21, 52.15it/s]\u001b[A\n",
      "28929it [10:21, 52.15it/s]\u001b[A\n",
      "28961it [10:22, 52.09it/s]\u001b[A\n",
      "28993it [10:23, 52.14it/s]\u001b[A\n",
      "29025it [10:23, 52.12it/s]\u001b[A\n",
      "29057it [10:24, 52.03it/s]\u001b[A\n",
      "29089it [10:24, 52.16it/s]\u001b[A\n",
      "29121it [10:25, 52.02it/s]\u001b[A\n",
      "29153it [10:26, 52.09it/s]\u001b[A\n",
      "29185it [10:26, 52.16it/s]\u001b[A\n",
      "29217it [10:27, 52.08it/s]\u001b[A\n",
      "29249it [10:28, 52.10it/s]\u001b[A\n",
      "29281it [10:28, 52.09it/s]\u001b[A\n",
      "29313it [10:29, 52.08it/s]\u001b[A\n",
      "29345it [10:29, 52.12it/s]\u001b[A\n",
      "29377it [10:30, 52.13it/s]\u001b[A\n",
      "29409it [10:31, 52.04it/s]\u001b[A\n",
      "29441it [10:31, 52.09it/s]\u001b[A\n",
      "29473it [10:32, 52.10it/s]\u001b[A\n",
      "29505it [10:32, 52.15it/s]\u001b[A\n",
      "29537it [10:33, 52.13it/s]\u001b[A\n",
      "29569it [10:34, 52.17it/s]\u001b[A\n",
      "29601it [10:34, 52.15it/s]\u001b[A\n",
      "29633it [10:35, 52.06it/s]\u001b[A\n",
      "29665it [10:36, 52.15it/s]\u001b[A\n",
      "29697it [10:36, 52.05it/s]\u001b[A\n",
      "29729it [10:37, 52.05it/s]\u001b[A\n",
      "29761it [10:37, 52.11it/s]\u001b[A\n",
      "29793it [10:38, 52.03it/s]\u001b[A\n",
      "29825it [10:39, 52.05it/s]\u001b[A\n",
      "29857it [10:39, 52.15it/s]\u001b[A\n",
      "29889it [10:40, 52.06it/s]\u001b[A\n",
      "29921it [10:40, 52.13it/s]\u001b[A\n",
      "29953it [10:41, 52.15it/s]\u001b[A\n",
      "29985it [10:42, 52.08it/s]\u001b[A\n",
      "30017it [10:42, 52.16it/s]\u001b[A\n",
      "30049it [10:43, 52.08it/s]\u001b[A\n",
      "30081it [10:44, 52.11it/s]\u001b[A\n",
      "30113it [10:44, 52.14it/s]\u001b[A\n",
      "30145it [10:45, 52.01it/s]\u001b[A\n",
      "30177it [10:45, 52.15it/s]\u001b[A\n",
      "30209it [10:46, 52.14it/s]\u001b[A\n",
      "30241it [10:47, 52.14it/s]\u001b[A\n",
      "30273it [10:47, 52.13it/s]\u001b[A\n",
      "30305it [10:48, 52.06it/s]\u001b[A\n",
      "30337it [10:48, 52.17it/s]\u001b[A\n",
      "30369it [10:49, 52.17it/s]\u001b[A\n",
      "30401it [10:50, 52.03it/s]\u001b[A\n",
      "30433it [10:50, 52.10it/s]\u001b[A\n",
      "30465it [10:51, 52.10it/s]\u001b[A\n",
      "30497it [10:52, 52.14it/s]\u001b[A\n",
      "30529it [10:52, 52.11it/s]\u001b[A\n",
      "30561it [10:53, 52.08it/s]\u001b[A\n",
      "30593it [10:53, 52.12it/s]\u001b[A\n",
      "30625it [10:54, 52.08it/s]\u001b[A\n",
      "30657it [10:55, 52.14it/s]\u001b[A\n",
      "30689it [10:55, 52.12it/s]\u001b[A\n",
      "30721it [10:56, 52.12it/s]\u001b[A\n",
      "30753it [10:56, 52.08it/s]\u001b[A\n",
      "30785it [10:57, 52.08it/s]\u001b[A\n",
      "30817it [10:58, 52.18it/s]\u001b[A\n",
      "30849it [10:58, 52.10it/s]\u001b[A\n",
      "30881it [10:59, 52.09it/s]\u001b[A\n",
      "30913it [10:59, 52.11it/s]\u001b[A\n",
      "30945it [11:00, 52.08it/s]\u001b[A\n",
      "30977it [11:01, 52.09it/s]\u001b[A\n",
      "31009it [11:01, 52.16it/s]\u001b[A\n",
      "31041it [11:02, 52.12it/s]\u001b[A\n",
      "31073it [11:03, 52.12it/s]\u001b[A\n",
      "31105it [11:03, 52.15it/s]\u001b[A\n",
      "31137it [11:04, 52.17it/s]\u001b[A\n",
      "31169it [11:04, 52.12it/s]\u001b[A\n",
      "31201it [11:05, 52.11it/s]\u001b[A\n",
      "31233it [11:06, 52.05it/s]\u001b[A\n",
      "31265it [11:06, 52.10it/s]\u001b[A\n",
      "31297it [11:07, 52.12it/s]\u001b[A\n",
      "31329it [11:07, 52.13it/s]\u001b[A\n",
      "31361it [11:08, 52.13it/s]\u001b[A\n",
      "31393it [11:09, 52.14it/s]\u001b[A\n",
      "31425it [11:09, 52.13it/s]\u001b[A\n",
      "31457it [11:10, 52.09it/s]\u001b[A\n",
      "31489it [11:11, 52.16it/s]\u001b[A\n",
      "31521it [11:11, 52.15it/s]\u001b[A\n",
      "31553it [11:12, 52.15it/s]\u001b[A\n",
      "31585it [11:12, 52.06it/s]\u001b[A\n",
      "31617it [11:13, 52.13it/s]\u001b[A\n",
      "31649it [11:14, 52.03it/s]\u001b[A\n",
      "31681it [11:14, 52.14it/s]\u001b[A\n",
      "31713it [11:15, 52.10it/s]\u001b[A\n",
      "31745it [11:15, 52.15it/s]\u001b[A\n",
      "31777it [11:16, 52.20it/s]\u001b[A\n",
      "31809it [11:17, 52.13it/s]\u001b[A\n",
      "31841it [11:17, 52.14it/s]\u001b[A\n",
      "31873it [11:18, 52.05it/s]\u001b[A\n",
      "31905it [11:19, 52.12it/s]\u001b[A\n",
      "31937it [11:19, 52.10it/s]\u001b[A\n",
      "31969it [11:20, 52.17it/s]\u001b[A\n",
      "32001it [11:20, 52.08it/s]\u001b[A\n",
      "32033it [11:21, 52.08it/s]\u001b[A\n",
      "32065it [11:22, 52.13it/s]\u001b[A\n",
      "32097it [11:22, 52.19it/s]\u001b[A\n",
      "32129it [11:23, 52.08it/s]\u001b[A\n",
      "32161it [11:23, 51.84it/s]\u001b[A\n",
      "32193it [11:24, 51.90it/s]\u001b[A\n",
      "32225it [11:25, 52.12it/s]\u001b[A\n",
      "32257it [11:25, 51.97it/s]\u001b[A\n",
      "32289it [11:26, 51.80it/s]\u001b[A\n",
      "32321it [11:27, 52.00it/s]\u001b[A\n",
      "32353it [11:27, 51.96it/s]\u001b[A\n",
      "32385it [11:28, 51.97it/s]\u001b[A\n",
      "32417it [11:28, 52.19it/s]\u001b[A\n",
      "32449it [11:29, 51.97it/s]\u001b[A\n",
      "32481it [11:30, 52.01it/s]\u001b[A\n",
      "32513it [11:30, 52.10it/s]\u001b[A\n",
      "32545it [11:31, 52.20it/s]\u001b[A\n",
      "32577it [11:31, 52.12it/s]\u001b[A\n",
      "32609it [11:32, 51.99it/s]\u001b[A\n",
      "32641it [11:33, 52.12it/s]\u001b[A\n",
      "32673it [11:33, 52.10it/s]\u001b[A\n",
      "32705it [11:34, 52.22it/s]\u001b[A\n",
      "32737it [11:34, 52.14it/s]\u001b[A\n",
      "32769it [11:35, 52.08it/s]\u001b[A\n",
      "32801it [11:36, 52.05it/s]\u001b[A\n",
      "32833it [11:36, 52.10it/s]\u001b[A\n",
      "32865it [11:37, 52.09it/s]\u001b[A\n",
      "32897it [11:38, 52.09it/s]\u001b[A\n",
      "32929it [11:38, 52.12it/s]\u001b[A\n",
      "32961it [11:39, 52.12it/s]\u001b[A\n",
      "32993it [11:39, 52.08it/s]\u001b[A\n",
      "33025it [11:40, 52.13it/s]\u001b[A\n",
      "33057it [11:41, 52.17it/s]\u001b[A\n",
      "33089it [11:41, 52.10it/s]\u001b[A\n",
      "33121it [11:42, 52.09it/s]\u001b[A\n",
      "33153it [11:42, 52.14it/s]\u001b[A\n",
      "33185it [11:43, 52.16it/s]\u001b[A\n",
      "33217it [11:44, 52.15it/s]\u001b[A\n",
      "33249it [11:44, 51.79it/s]\u001b[A\n",
      "33281it [11:45, 51.79it/s]\u001b[A\n",
      "33313it [11:46, 51.77it/s]\u001b[A\n",
      "33345it [11:46, 51.87it/s]\u001b[A\n",
      "33377it [11:47, 51.92it/s]\u001b[A\n",
      "33409it [11:47, 51.95it/s]\u001b[A\n",
      "33441it [11:48, 52.05it/s]\u001b[A\n",
      "33473it [11:49, 51.99it/s]\u001b[A\n",
      "33505it [11:49, 51.78it/s]\u001b[A\n",
      "33537it [11:50, 51.67it/s]\u001b[A\n",
      "33569it [11:51, 51.79it/s]\u001b[A\n",
      "33601it [11:51, 51.94it/s]\u001b[A\n",
      "33633it [11:52, 51.91it/s]\u001b[A\n",
      "33665it [11:52, 52.01it/s]\u001b[A\n",
      "33697it [11:53, 52.05it/s]\u001b[A\n",
      "33729it [11:54, 52.00it/s]\u001b[A\n",
      "33761it [11:54, 52.10it/s]\u001b[A\n",
      "33793it [11:55, 52.08it/s]\u001b[A\n",
      "33825it [11:55, 52.12it/s]\u001b[A\n",
      "33857it [11:56, 52.13it/s]\u001b[A\n",
      "33889it [11:57, 52.24it/s]\u001b[A\n",
      "33921it [11:57, 52.42it/s]\u001b[A\n",
      "33953it [11:58, 52.37it/s]\u001b[A\n",
      "33985it [11:58, 52.38it/s]\u001b[A\n",
      "34017it [11:59, 52.26it/s]\u001b[A\n",
      "34049it [12:00, 52.15it/s]\u001b[A\n",
      "34081it [12:00, 52.12it/s]\u001b[A\n",
      "34113it [12:01, 52.24it/s]\u001b[A\n",
      "34145it [12:02, 52.18it/s]\u001b[A\n",
      "34177it [12:02, 52.13it/s]\u001b[A\n",
      "34209it [12:03, 52.12it/s]\u001b[A\n",
      "34241it [12:03, 52.20it/s]\u001b[A\n",
      "34273it [12:04, 52.13it/s]\u001b[A\n",
      "34305it [12:05, 52.07it/s]\u001b[A\n",
      "34337it [12:05, 52.15it/s]\u001b[A\n",
      "34369it [12:06, 52.14it/s]\u001b[A\n",
      "34401it [12:06, 52.20it/s]\u001b[A\n",
      "34433it [12:07, 52.11it/s]\u001b[A\n",
      "34465it [12:08, 52.13it/s]\u001b[A\n",
      "34497it [12:08, 52.13it/s]\u001b[A\n",
      "34529it [12:09, 52.22it/s]\u001b[A\n",
      "34561it [12:10, 52.44it/s]\u001b[A\n",
      "34593it [12:10, 52.47it/s]\u001b[A\n",
      "34625it [12:11, 52.26it/s]\u001b[A\n",
      "34657it [12:11, 52.30it/s]\u001b[A\n",
      "34689it [12:12, 52.27it/s]\u001b[A\n",
      "34721it [12:13, 52.15it/s]\u001b[A\n",
      "34753it [12:13, 52.22it/s]\u001b[A\n",
      "34785it [12:14, 52.17it/s]\u001b[A\n",
      "34817it [12:14, 52.17it/s]\u001b[A\n",
      "34849it [12:15, 52.16it/s]\u001b[A\n",
      "34881it [12:16, 52.09it/s]\u001b[A\n",
      "34913it [12:16, 52.16it/s]\u001b[A\n",
      "34945it [12:17, 52.18it/s]\u001b[A\n",
      "34977it [12:17, 52.15it/s]\u001b[A\n",
      "35009it [12:18, 52.11it/s]\u001b[A\n",
      "35041it [12:19, 52.09it/s]\u001b[A\n",
      "35073it [12:19, 52.12it/s]\u001b[A\n",
      "35105it [12:20, 52.14it/s]\u001b[A\n",
      "35137it [12:21, 52.13it/s]\u001b[A\n",
      "35169it [12:21, 52.13it/s]\u001b[A\n",
      "35201it [12:22, 52.10it/s]\u001b[A\n",
      "35233it [12:22, 52.15it/s]\u001b[A\n",
      "35265it [12:23, 52.13it/s]\u001b[A\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "35297it [12:24, 52.09it/s]\u001b[A\n",
      "35329it [12:24, 52.11it/s]\u001b[A\n",
      "35361it [12:25, 52.05it/s]\u001b[A\n",
      "35393it [12:25, 52.10it/s]\u001b[A\n",
      "35425it [12:26, 52.09it/s]\u001b[A\n",
      "35457it [12:27, 52.13it/s]\u001b[A\n",
      "35489it [12:27, 52.05it/s]\u001b[A\n",
      "35521it [12:28, 52.11it/s]\u001b[A\n",
      "35553it [12:29, 52.27it/s]\u001b[A\n",
      "35585it [12:29, 51.85it/s]\u001b[A\n",
      "35617it [12:30, 52.17it/s]\u001b[A\n",
      "35649it [12:30, 52.12it/s]\u001b[A\n",
      "35681it [12:31, 52.18it/s]\u001b[A\n",
      "35713it [12:32, 52.10it/s]\u001b[A\n",
      "35745it [12:32, 52.14it/s]\u001b[A\n",
      "35777it [12:33, 52.03it/s]\u001b[A\n",
      "35809it [12:33, 52.10it/s]\u001b[A\n",
      "35841it [12:34, 52.17it/s]\u001b[A\n",
      "35873it [12:35, 52.09it/s]\u001b[A\n",
      "35905it [12:35, 52.20it/s]\u001b[A\n",
      "35937it [12:36, 52.15it/s]\u001b[A\n",
      "35969it [12:37, 52.05it/s]\u001b[A\n",
      "36001it [12:37, 52.08it/s]\u001b[A\n",
      "36033it [12:38, 52.08it/s]\u001b[A\n",
      "36065it [12:38, 52.05it/s]\u001b[A\n",
      "36097it [12:39, 52.06it/s]\u001b[A\n",
      "36129it [12:40, 52.09it/s]\u001b[A\n",
      "36161it [12:40, 52.11it/s]\u001b[A\n",
      "36193it [12:41, 52.12it/s]\u001b[A\n",
      "36225it [12:41, 52.16it/s]\u001b[A\n",
      "36257it [12:42, 52.12it/s]\u001b[A\n",
      "36289it [12:43, 52.02it/s]\u001b[A\n",
      "36321it [12:43, 52.05it/s]\u001b[A\n",
      "36353it [12:44, 52.14it/s]\u001b[A\n",
      "36385it [12:45, 52.12it/s]\u001b[A\n",
      "36417it [12:45, 52.13it/s]\u001b[A\n",
      "36449it [12:46, 52.12it/s]\u001b[A\n",
      "36481it [12:46, 52.18it/s]\u001b[A\n",
      "36513it [12:47, 52.07it/s]\u001b[A\n",
      "36545it [12:48, 52.12it/s]\u001b[A\n",
      "36577it [12:48, 52.15it/s]\u001b[A\n",
      "36609it [12:49, 52.09it/s]\u001b[A\n",
      "36641it [12:49, 52.12it/s]\u001b[A\n",
      "36673it [12:50, 52.27it/s]\u001b[A\n",
      "36705it [12:51, 52.09it/s]\u001b[A\n",
      "36737it [12:51, 52.08it/s]\u001b[A\n",
      "36769it [12:52, 52.07it/s]\u001b[A\n",
      "36801it [12:52, 52.16it/s]\u001b[A\n",
      "36833it [12:53, 52.14it/s]\u001b[A\n",
      "36865it [12:54, 52.17it/s]\u001b[A\n",
      "36897it [12:54, 52.09it/s]\u001b[A\n",
      "36929it [12:55, 52.16it/s]\u001b[A\n",
      "36961it [12:56, 52.00it/s]\u001b[A\n",
      "36993it [12:56, 52.15it/s]\u001b[A\n",
      "37025it [12:57, 52.05it/s]\u001b[A\n",
      "37057it [12:57, 52.17it/s]\u001b[A\n",
      "37089it [12:58, 52.12it/s]\u001b[A\n",
      "37121it [12:59, 52.10it/s]\u001b[A\n",
      "37153it [12:59, 52.15it/s]\u001b[A\n",
      "37185it [13:00, 52.07it/s]\u001b[A\n",
      "37217it [13:00, 52.13it/s]\u001b[A\n",
      "37249it [13:01, 52.20it/s]\u001b[A\n",
      "37281it [13:02, 52.07it/s]\u001b[A\n",
      "37313it [13:02, 52.15it/s]\u001b[A\n",
      "37345it [13:03, 52.14it/s]\u001b[A\n",
      "37377it [13:04, 52.15it/s]\u001b[A\n",
      "37409it [13:04, 52.10it/s]\u001b[A\n",
      "37441it [13:05, 52.10it/s]\u001b[A\n",
      "37473it [13:05, 52.09it/s]\u001b[A\n",
      "37505it [13:06, 52.16it/s]\u001b[A\n",
      "37537it [13:07, 52.13it/s]\u001b[A\n",
      "37569it [13:07, 52.17it/s]\u001b[A\n",
      "37601it [13:08, 52.18it/s]\u001b[A\n",
      "37633it [13:08, 52.14it/s]\u001b[A\n",
      "37665it [13:09, 52.11it/s]\u001b[A\n",
      "37697it [13:10, 52.14it/s]\u001b[A\n",
      "37729it [13:10, 52.12it/s]\u001b[A\n",
      "37761it [13:11, 52.13it/s]\u001b[A\n",
      "37793it [13:12, 52.10it/s]\u001b[A\n",
      "37825it [13:12, 52.21it/s]\u001b[A\n",
      "37857it [13:13, 52.12it/s]\u001b[A\n",
      "37889it [13:13, 52.06it/s]\u001b[A\n",
      "37921it [13:14, 52.09it/s]\u001b[A\n",
      "37953it [13:15, 52.09it/s]\u001b[A\n",
      "37985it [13:15, 52.08it/s]\u001b[A\n",
      "38017it [13:16, 52.14it/s]\u001b[A\n",
      "38049it [13:16, 52.15it/s]\u001b[A\n",
      "38081it [13:17, 52.10it/s]\u001b[A\n",
      "38113it [13:18, 52.08it/s]\u001b[A\n",
      "38145it [13:18, 52.05it/s]\u001b[A\n",
      "38177it [13:19, 52.12it/s]\u001b[A\n",
      "38209it [13:20, 52.18it/s]\u001b[A\n",
      "38241it [13:20, 52.05it/s]\u001b[A\n",
      "38273it [13:21, 52.05it/s]\u001b[A\n",
      "38305it [13:21, 52.09it/s]\u001b[A\n",
      "38337it [13:22, 52.07it/s]\u001b[A\n",
      "38369it [13:23, 52.09it/s]\u001b[A\n",
      "38401it [13:23, 52.16it/s]\u001b[A\n",
      "38433it [13:24, 52.11it/s]\u001b[A\n",
      "38465it [13:24, 52.12it/s]\u001b[A\n",
      "38497it [13:25, 52.12it/s]\u001b[A\n",
      "38529it [13:26, 52.06it/s]\u001b[A\n",
      "38561it [13:26, 52.12it/s]\u001b[A\n",
      "38593it [13:27, 52.08it/s]\u001b[A\n",
      "38625it [13:27, 51.93it/s]\u001b[A\n",
      "38657it [13:28, 52.15it/s]\u001b[A\n",
      "38689it [13:29, 52.10it/s]\u001b[A\n",
      "38721it [13:29, 52.13it/s]\u001b[A\n",
      "38753it [13:30, 52.12it/s]\u001b[A\n",
      "38785it [13:31, 52.07it/s]\u001b[A\n",
      "38817it [13:31, 52.09it/s]\u001b[A\n",
      "38849it [13:32, 52.08it/s]\u001b[A\n",
      "38881it [13:32, 52.28it/s]\u001b[A\n",
      "38913it [13:33, 52.17it/s]\u001b[A\n",
      "38945it [13:34, 52.05it/s]\u001b[A\n",
      "38977it [13:34, 52.07it/s]\u001b[A\n",
      "39009it [13:35, 52.07it/s]\u001b[A\n",
      "39041it [13:35, 52.06it/s]\u001b[A\n",
      "39073it [13:36, 52.13it/s]\u001b[A\n",
      "39105it [13:37, 51.97it/s]\u001b[A\n",
      "39137it [13:37, 52.17it/s]\u001b[A\n",
      "39169it [13:38, 52.17it/s]\u001b[A\n",
      "39201it [13:39, 52.15it/s]\u001b[A\n",
      "39233it [13:39, 52.16it/s]\u001b[A\n",
      "39265it [13:40, 52.11it/s]\u001b[A\n",
      "39297it [13:40, 52.09it/s]\u001b[A\n",
      "39329it [13:41, 52.13it/s]\u001b[A\n",
      "39361it [13:42, 51.99it/s]\u001b[A\n",
      "39393it [13:42, 52.17it/s]\u001b[A\n",
      "39425it [13:43, 52.15it/s]\u001b[A\n",
      "39457it [13:43, 52.11it/s]\u001b[A\n",
      "39489it [13:44, 52.18it/s]\u001b[A\n",
      "39521it [13:45, 52.12it/s]\u001b[A\n",
      "39553it [13:45, 52.19it/s]\u001b[A\n",
      "39585it [13:46, 52.12it/s]\u001b[A\n",
      "39617it [13:47, 52.14it/s]\u001b[A\n",
      "39649it [13:47, 52.09it/s]\u001b[A\n",
      "39681it [13:48, 52.10it/s]\u001b[A\n",
      "39713it [13:48, 52.13it/s]\u001b[A\n",
      "39745it [13:49, 52.13it/s]\u001b[A\n",
      "39777it [13:50, 52.09it/s]\u001b[A\n",
      "39809it [13:50, 52.07it/s]\u001b[A\n",
      "39841it [13:51, 52.12it/s]\u001b[A\n",
      "39873it [13:51, 52.23it/s]\u001b[A\n",
      "39905it [13:52, 52.13it/s]\u001b[A\n",
      "39937it [13:53, 52.05it/s]\u001b[A\n",
      "39969it [13:53, 52.03it/s]\u001b[A\n",
      "40001it [13:54, 52.04it/s]\u001b[A\n",
      "40033it [13:55, 52.08it/s]\u001b[A\n",
      "40065it [13:55, 52.14it/s]\u001b[A\n",
      "40097it [13:56, 52.12it/s]\u001b[A\n",
      "40129it [13:56, 52.03it/s]\u001b[A\n",
      "40161it [13:57, 52.12it/s]\u001b[A\n",
      "40193it [13:58, 52.12it/s]\u001b[A\n",
      "40225it [13:58, 52.15it/s]\u001b[A\n",
      "40257it [13:59, 52.15it/s]\u001b[A\n",
      "40289it [13:59, 52.08it/s]\u001b[A\n",
      "40321it [14:00, 52.15it/s]\u001b[A\n",
      "40353it [14:01, 52.12it/s]\u001b[A\n",
      "40385it [14:01, 52.13it/s]\u001b[A\n",
      "40417it [14:02, 52.11it/s]\u001b[A\n",
      "40449it [14:02, 52.08it/s]\u001b[A\n",
      "40481it [14:03, 52.08it/s]\u001b[A\n",
      "40513it [14:04, 52.17it/s]\u001b[A\n",
      "40545it [14:04, 52.15it/s]\u001b[A\n",
      "40577it [14:05, 52.13it/s]\u001b[A\n",
      "40609it [14:06, 52.06it/s]\u001b[A\n",
      "40641it [14:06, 52.08it/s]\u001b[A\n",
      "40673it [14:07, 52.26it/s]\u001b[A\n",
      "40705it [14:07, 52.15it/s]\u001b[A\n",
      "40737it [14:08, 52.11it/s]\u001b[A\n",
      "40769it [14:09, 52.10it/s]\u001b[A\n",
      "40801it [14:09, 52.11it/s]\u001b[A\n",
      "40833it [14:10, 52.05it/s]\u001b[A\n",
      "40865it [14:10, 52.13it/s]\u001b[A\n",
      "40897it [14:11, 52.10it/s]\u001b[A\n",
      "40929it [14:12, 52.12it/s]\u001b[A\n",
      "40961it [14:12, 52.15it/s]\u001b[A\n",
      "40993it [14:13, 52.14it/s]\u001b[A\n",
      "41025it [14:14, 52.14it/s]\u001b[A\n",
      "41057it [14:14, 52.08it/s]\u001b[A\n",
      "41089it [14:15, 52.12it/s]\u001b[A\n",
      "41121it [14:15, 52.11it/s]\u001b[A\n",
      "41153it [14:16, 52.11it/s]\u001b[A\n",
      "41185it [14:17, 52.12it/s]\u001b[A\n",
      "41217it [14:17, 52.11it/s]\u001b[A\n",
      "41249it [14:18, 52.12it/s]\u001b[A\n",
      "41281it [14:18, 52.03it/s]\u001b[A\n",
      "41313it [14:19, 52.12it/s]\u001b[A\n",
      "41345it [14:20, 52.11it/s]\u001b[A\n",
      "41377it [14:20, 52.15it/s]\u001b[A\n",
      "41409it [14:21, 52.12it/s]\u001b[A\n",
      "41441it [14:22, 52.14it/s]\u001b[A\n",
      "41473it [14:22, 52.10it/s]\u001b[A\n",
      "41505it [14:23, 52.14it/s]\u001b[A\n",
      "41537it [14:23, 52.21it/s]\u001b[A\n",
      "41569it [14:24, 52.05it/s]\u001b[A\n",
      "41601it [14:25, 52.07it/s]\u001b[A\n",
      "41633it [14:25, 52.06it/s]\u001b[A\n",
      "41665it [14:26, 52.06it/s]\u001b[A\n",
      "41697it [14:26, 51.98it/s]\u001b[A\n",
      "41729it [14:27, 52.09it/s]\u001b[A\n",
      "41761it [14:28, 52.21it/s]\u001b[A\n",
      "41793it [14:28, 52.13it/s]\u001b[A\n",
      "41825it [14:29, 52.07it/s]\u001b[A\n",
      "41857it [14:30, 52.06it/s]\u001b[A\n",
      "41889it [14:30, 52.16it/s]\u001b[A\n",
      "41921it [14:31, 52.18it/s]\u001b[A\n",
      "41953it [14:31, 52.15it/s]\u001b[A\n",
      "41985it [14:32, 52.09it/s]\u001b[A\n",
      "42017it [14:33, 52.15it/s]\u001b[A\n",
      "42049it [14:33, 52.06it/s]\u001b[A\n",
      "42081it [14:34, 52.02it/s]\u001b[A\n",
      "42113it [14:34, 52.15it/s]\u001b[A\n",
      "42145it [14:35, 52.17it/s]\u001b[A\n",
      "42177it [14:36, 52.15it/s]\u001b[A\n",
      "42209it [14:36, 52.14it/s]\u001b[A\n",
      "42241it [14:37, 52.09it/s]\u001b[A\n",
      "42273it [14:37, 52.04it/s]\u001b[A\n",
      "42305it [14:38, 52.17it/s]\u001b[A\n",
      "42337it [14:39, 52.15it/s]\u001b[A\n",
      "42369it [14:39, 52.15it/s]\u001b[A\n",
      "42401it [14:40, 52.12it/s]\u001b[A\n",
      "42433it [14:41, 52.12it/s]\u001b[A\n",
      "42465it [14:41, 52.10it/s]\u001b[A\n",
      "42497it [14:42, 52.10it/s]\u001b[A\n",
      "42529it [14:42, 52.09it/s]\u001b[A\n",
      "42561it [14:43, 52.09it/s]\u001b[A\n",
      "42593it [14:44, 52.12it/s]\u001b[A\n",
      "42625it [14:44, 52.10it/s]\u001b[A\n",
      "42657it [14:45, 52.11it/s]\u001b[A\n",
      "42689it [14:45, 52.13it/s]\u001b[A\n",
      "42721it [14:46, 52.05it/s]\u001b[A\n",
      "42753it [14:47, 52.07it/s]\u001b[A\n",
      "42785it [14:47, 52.03it/s]\u001b[A\n",
      "42817it [14:48, 52.11it/s]\u001b[A\n",
      "42849it [14:49, 52.09it/s]\u001b[A\n",
      "42881it [14:49, 52.06it/s]\u001b[A\n",
      "42913it [14:50, 52.13it/s]\u001b[A\n",
      "42945it [14:50, 51.97it/s]\u001b[A\n",
      "42977it [14:51, 52.15it/s]\u001b[A\n",
      "43009it [14:52, 52.13it/s]\u001b[A\n",
      "43041it [14:52, 52.10it/s]\u001b[A\n",
      "43073it [14:53, 52.10it/s]\u001b[A\n",
      "43105it [14:53, 52.11it/s]\u001b[A\n",
      "43137it [14:54, 52.03it/s]\u001b[A\n",
      "43169it [14:55, 52.06it/s]\u001b[A\n",
      "43201it [14:55, 52.11it/s]\u001b[A\n",
      "43233it [14:56, 52.07it/s]\u001b[A\n",
      "43265it [14:57, 52.07it/s]\u001b[A\n",
      "43297it [14:57, 52.10it/s]\u001b[A\n",
      "43329it [14:58, 52.11it/s]\u001b[A\n",
      "43361it [14:58, 52.25it/s]\u001b[A\n",
      "43393it [14:59, 52.09it/s]\u001b[A\n",
      "43425it [15:00, 52.07it/s]\u001b[A\n",
      "43457it [15:00, 52.11it/s]\u001b[A\n",
      "43489it [15:01, 52.14it/s]\u001b[A\n",
      "43521it [15:01, 52.16it/s]\u001b[A\n",
      "43553it [15:02, 52.08it/s]\u001b[A\n",
      "43585it [15:03, 52.08it/s]\u001b[A\n",
      "43617it [15:03, 51.91it/s]\u001b[A\n",
      "43649it [15:04, 52.12it/s]\u001b[A\n",
      "43681it [15:05, 52.15it/s]\u001b[A\n",
      "43713it [15:05, 52.19it/s]\u001b[A\n",
      "43745it [15:06, 52.14it/s]\u001b[A\n",
      "43777it [15:06, 52.15it/s]\u001b[A\n",
      "43809it [15:07, 51.84it/s]\u001b[A\n",
      "43841it [15:08, 51.69it/s]\u001b[A\n",
      "43873it [15:08, 51.87it/s]\u001b[A\n",
      "43905it [15:09, 51.87it/s]\u001b[A\n",
      "43937it [15:09, 51.87it/s]\u001b[A\n",
      "43969it [15:10, 52.01it/s]\u001b[A\n",
      "44001it [15:11, 51.92it/s]\u001b[A\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "44033it [15:11, 52.01it/s]\u001b[A\n",
      "44065it [15:12, 52.08it/s]\u001b[A\n",
      "44097it [15:13, 51.97it/s]\u001b[A\n",
      "44129it [15:13, 52.10it/s]\u001b[A\n",
      "44161it [15:14, 52.14it/s]\u001b[A\n",
      "44193it [15:14, 52.16it/s]\u001b[A\n",
      "44225it [15:15, 52.12it/s]\u001b[A\n",
      "44257it [15:16, 52.10it/s]\u001b[A\n",
      "44289it [15:16, 52.16it/s]\u001b[A\n",
      "44321it [15:17, 52.15it/s]\u001b[A\n",
      "44353it [15:17, 52.11it/s]\u001b[A\n",
      "44385it [15:18, 52.13it/s]\u001b[A\n",
      "44417it [15:19, 51.98it/s]\u001b[A\n",
      "44449it [15:19, 52.17it/s]\u001b[A\n",
      "44481it [15:20, 52.12it/s]\u001b[A\n",
      "44513it [15:21, 52.14it/s]\u001b[A\n",
      "44545it [15:21, 52.26it/s]\u001b[A\n",
      "44577it [15:22, 52.11it/s]\u001b[A\n",
      "44609it [15:22, 52.11it/s]\u001b[A\n",
      "44641it [15:23, 52.14it/s]\u001b[A\n",
      "44673it [15:24, 52.12it/s]\u001b[A\n",
      "44705it [15:24, 52.11it/s]\u001b[A\n",
      "44737it [15:25, 52.10it/s]\u001b[A\n",
      "44769it [15:25, 52.15it/s]\u001b[A\n",
      "44801it [15:26, 52.07it/s]\u001b[A\n",
      "44833it [15:27, 52.09it/s]\u001b[A\n",
      "44865it [15:27, 52.05it/s]\u001b[A\n",
      "44897it [15:28, 52.11it/s]\u001b[A\n",
      "44929it [15:28, 52.13it/s]\u001b[A\n",
      "44961it [15:29, 52.12it/s]\u001b[A\n",
      "44993it [15:30, 52.09it/s]\u001b[A\n",
      "45025it [15:30, 52.08it/s]\u001b[A\n",
      "45057it [15:31, 52.19it/s]\u001b[A\n",
      "45089it [15:32, 52.10it/s]\u001b[A\n",
      "45121it [15:32, 52.11it/s]\u001b[A\n",
      "45153it [15:33, 52.18it/s]\u001b[A\n",
      "45185it [15:33, 52.13it/s]\u001b[A\n",
      "45217it [15:34, 52.11it/s]\u001b[A\n",
      "45249it [15:35, 52.11it/s]\u001b[A\n",
      "45281it [15:35, 52.06it/s]\u001b[A\n",
      "45313it [15:36, 52.12it/s]\u001b[A\n",
      "45345it [15:36, 52.15it/s]\u001b[A\n",
      "45377it [15:37, 52.13it/s]\u001b[A\n",
      "45409it [15:38, 52.09it/s]\u001b[A\n",
      "45441it [15:38, 52.04it/s]\u001b[A\n",
      "45473it [15:39, 52.14it/s]\u001b[A\n",
      "45505it [15:40, 52.17it/s]\u001b[A\n",
      "45537it [15:40, 52.11it/s]\u001b[A\n",
      "45569it [15:41, 52.10it/s]\u001b[A\n",
      "45601it [15:41, 52.07it/s]\u001b[A\n",
      "45633it [15:42, 52.14it/s]\u001b[A\n",
      "45665it [15:43, 52.07it/s]\u001b[A\n",
      "45697it [15:43, 52.11it/s]\u001b[A\n",
      "45729it [15:44, 52.13it/s]\u001b[A\n",
      "45761it [15:44, 52.11it/s]\u001b[A\n",
      "45793it [15:45, 52.08it/s]\u001b[A\n",
      "45825it [15:46, 52.16it/s]\u001b[A\n",
      "45857it [15:46, 52.13it/s]\u001b[A\n",
      "45889it [15:47, 52.11it/s]\u001b[A\n",
      "45921it [15:48, 52.16it/s]\u001b[A\n",
      "45953it [15:48, 52.03it/s]\u001b[A\n",
      "45985it [15:49, 52.09it/s]\u001b[A\n",
      "46017it [15:49, 52.16it/s]\u001b[A\n",
      "46049it [15:50, 52.13it/s]\u001b[A\n",
      "46081it [15:51, 52.29it/s]\u001b[A\n",
      "46113it [15:51, 52.11it/s]\u001b[A\n",
      "46145it [15:52, 52.05it/s]\u001b[A\n",
      "46177it [15:52, 52.07it/s]\u001b[A\n",
      "46209it [15:53, 52.09it/s]\u001b[A\n",
      "46241it [15:54, 52.09it/s]\u001b[A\n",
      "46273it [15:54, 52.10it/s]\u001b[A\n",
      "46305it [15:55, 52.14it/s]\u001b[A\n",
      "46337it [15:56, 52.08it/s]\u001b[A\n",
      "46369it [15:56, 52.03it/s]\u001b[A\n",
      "46401it [15:57, 52.13it/s]\u001b[A\n",
      "46433it [15:57, 52.11it/s]\u001b[A\n",
      "46465it [15:58, 52.08it/s]\u001b[A\n",
      "46497it [15:59, 52.13it/s]\u001b[A\n",
      "46529it [15:59, 52.14it/s]\u001b[A\n",
      "46561it [16:00, 52.08it/s]\u001b[A\n",
      "46593it [16:00, 52.15it/s]\u001b[A\n",
      "46625it [16:01, 52.08it/s]\u001b[A\n",
      "46657it [16:02, 52.07it/s]\u001b[A\n",
      "46689it [16:02, 52.05it/s]\u001b[A\n",
      "46721it [16:03, 52.02it/s]\u001b[A\n",
      "46753it [16:03, 52.15it/s]\u001b[A\n",
      "46785it [16:04, 52.18it/s]\u001b[A\n",
      "46817it [16:05, 52.09it/s]\u001b[A\n",
      "46849it [16:05, 52.06it/s]\u001b[A\n",
      "46881it [16:06, 52.14it/s]\u001b[A\n",
      "46913it [16:07, 52.13it/s]\u001b[A\n",
      "46945it [16:07, 52.08it/s]\u001b[A\n",
      "46977it [16:08, 52.09it/s]\u001b[A\n",
      "47009it [16:08, 52.16it/s]\u001b[A\n",
      "47041it [16:09, 52.13it/s]\u001b[A\n",
      "47073it [16:10, 52.29it/s]\u001b[A\n",
      "47105it [16:10, 52.30it/s]\u001b[A\n",
      "47137it [16:11, 52.48it/s]\u001b[A\n",
      "47169it [16:11, 52.42it/s]\u001b[A\n",
      "47201it [16:12, 52.31it/s]\u001b[A\n",
      "47233it [16:13, 52.24it/s]\u001b[A\n",
      "47265it [16:13, 52.20it/s]\u001b[A\n",
      "47297it [16:14, 52.07it/s]\u001b[A\n",
      "47329it [16:15, 52.13it/s]\u001b[A\n",
      "47361it [16:15, 52.17it/s]\u001b[A\n",
      "47393it [16:16, 52.09it/s]\u001b[A\n",
      "47425it [16:16, 52.14it/s]\u001b[A\n",
      "47457it [16:17, 52.05it/s]\u001b[A\n",
      "47489it [16:18, 52.13it/s]\u001b[A\n",
      "47521it [16:18, 52.17it/s]\u001b[A\n",
      "47553it [16:19, 52.10it/s]\u001b[A\n",
      "47585it [16:19, 52.14it/s]\u001b[A\n",
      "47617it [16:20, 52.11it/s]\u001b[A\n",
      "47649it [16:21, 52.06it/s]\u001b[A\n",
      "47681it [16:21, 52.09it/s]\u001b[A\n",
      "47713it [16:22, 52.16it/s]\u001b[A\n",
      "47745it [16:23, 52.14it/s]\u001b[A\n",
      "47777it [16:23, 52.12it/s]\u001b[A\n",
      "47809it [16:24, 52.06it/s]\u001b[A\n",
      "47841it [16:24, 52.11it/s]\u001b[A\n",
      "47873it [16:25, 52.12it/s]\u001b[A\n",
      "47905it [16:26, 52.07it/s]\u001b[A\n",
      "47937it [16:26, 52.09it/s]\u001b[A\n",
      "47969it [16:27, 52.10it/s]\u001b[A\n",
      "48001it [16:27, 52.15it/s]\u001b[A\n",
      "48033it [16:28, 52.11it/s]\u001b[A\n",
      "48065it [16:29, 52.14it/s]\u001b[A\n",
      "48097it [16:29, 52.15it/s]\u001b[A\n",
      "48129it [16:30, 52.07it/s]\u001b[A\n",
      "48161it [16:30, 52.03it/s]\u001b[A\n",
      "48193it [16:31, 52.05it/s]\u001b[A\n",
      "48225it [16:32, 52.14it/s]\u001b[A\n",
      "48257it [16:32, 52.08it/s]\u001b[A\n",
      "48289it [16:33, 52.14it/s]\u001b[A\n",
      "48321it [16:34, 52.12it/s]\u001b[A\n",
      "48353it [16:34, 52.13it/s]\u001b[A\n",
      "48385it [16:35, 52.17it/s]\u001b[A\n",
      "48417it [16:35, 52.13it/s]\u001b[A\n",
      "48449it [16:36, 52.17it/s]\u001b[A\n",
      "48481it [16:37, 52.05it/s]\u001b[A\n",
      "48513it [16:37, 52.30it/s]\u001b[A\n",
      "48545it [16:38, 52.01it/s]\u001b[A\n",
      "48577it [16:38, 52.06it/s]\u001b[A\n",
      "48609it [16:39, 52.09it/s]\u001b[A\n",
      "48641it [16:40, 52.10it/s]\u001b[A\n",
      "48673it [16:40, 52.08it/s]\u001b[A\n",
      "48705it [16:41, 52.03it/s]\u001b[A\n",
      "48737it [16:42, 52.04it/s]\u001b[A\n",
      "48769it [16:42, 52.11it/s]\u001b[A\n",
      "48801it [16:43, 52.15it/s]\u001b[A\n",
      "48833it [16:43, 52.07it/s]\u001b[A\n",
      "48865it [16:44, 52.09it/s]\u001b[A\n",
      "48897it [16:45, 52.06it/s]\u001b[A\n",
      "48929it [16:45, 52.16it/s]\u001b[A\n",
      "48961it [16:46, 52.10it/s]\u001b[A\n",
      "48993it [16:46, 52.01it/s]\u001b[A\n",
      "49025it [16:47, 52.13it/s]\u001b[A\n",
      "49057it [16:48, 52.11it/s]\u001b[A\n",
      "49089it [16:48, 52.24it/s]\u001b[A\n",
      "49121it [16:49, 52.17it/s]\u001b[A\n",
      "49153it [16:50, 52.06it/s]\u001b[A\n",
      "49185it [16:50, 52.13it/s]\u001b[A\n",
      "49217it [16:51, 52.09it/s]\u001b[A\n",
      "49249it [16:51, 52.11it/s]\u001b[A\n",
      "49281it [16:52, 52.17it/s]\u001b[A\n",
      "49313it [16:53, 52.11it/s]\u001b[A\n",
      "49345it [16:53, 52.03it/s]\u001b[A\n",
      "49377it [16:54, 52.11it/s]\u001b[A\n",
      "49409it [16:54, 52.17it/s]\u001b[A\n",
      "49441it [16:55, 52.05it/s]\u001b[A\n",
      "49473it [16:56, 52.02it/s]\u001b[A\n",
      "49505it [16:56, 51.99it/s]\u001b[A\n",
      "49537it [16:57, 52.04it/s]\u001b[A\n",
      "49569it [16:58, 52.09it/s]\u001b[A\n",
      "49601it [16:58, 52.11it/s]\u001b[A\n",
      "49633it [16:59, 52.14it/s]\u001b[A\n",
      "49665it [16:59, 52.12it/s]\u001b[A\n",
      "49697it [17:00, 52.11it/s]\u001b[A\n",
      "49729it [17:01, 52.11it/s]\u001b[A\n",
      "49761it [17:01, 52.22it/s]\u001b[A\n",
      "49793it [17:02, 52.03it/s]\u001b[A\n",
      "49825it [17:02, 52.07it/s]\u001b[A\n",
      "49857it [17:03, 52.09it/s]\u001b[A\n",
      "49889it [17:04, 52.09it/s]\u001b[A\n",
      "49921it [17:04, 52.14it/s]\u001b[A\n",
      "49953it [17:05, 52.18it/s]\u001b[A\n",
      "49985it [17:05, 52.18it/s]\u001b[A\n",
      "50017it [17:06, 52.10it/s]\u001b[A\n",
      "50049it [17:07, 52.17it/s]\u001b[A\n",
      "50081it [17:07, 52.12it/s]\u001b[A\n",
      "50113it [17:08, 52.15it/s]\u001b[A\n",
      "50145it [17:09, 52.13it/s]\u001b[A\n",
      "50177it [17:09, 52.07it/s]\u001b[A\n",
      "50209it [17:10, 52.11it/s]\u001b[A\n",
      "50241it [17:10, 52.00it/s]\u001b[A\n",
      "50273it [17:11, 52.13it/s]\u001b[A\n",
      "50305it [17:12, 52.10it/s]\u001b[A\n",
      "50337it [17:12, 52.22it/s]\u001b[A\n",
      "50369it [17:13, 52.06it/s]\u001b[A\n",
      "50401it [17:13, 52.08it/s]\u001b[A\n",
      "50433it [17:14, 52.16it/s]\u001b[A\n",
      "50465it [17:15, 52.17it/s]\u001b[A\n",
      "50497it [17:15, 52.06it/s]\u001b[A\n",
      "50529it [17:16, 52.12it/s]\u001b[A\n",
      "50561it [17:17, 52.11it/s]\u001b[A\n",
      "50593it [17:17, 52.08it/s]\u001b[A\n",
      "50625it [17:18, 52.18it/s]\u001b[A\n",
      "50657it [17:18, 52.02it/s]\u001b[A\n",
      "50689it [17:19, 52.12it/s]\u001b[A\n",
      "50721it [17:20, 52.13it/s]\u001b[A\n",
      "50753it [17:20, 52.15it/s]\u001b[A\n",
      "50785it [17:21, 52.12it/s]\u001b[A\n",
      "50817it [17:21, 52.10it/s]\u001b[A\n",
      "50849it [17:22, 52.14it/s]\u001b[A\n",
      "50881it [17:23, 52.12it/s]\u001b[A\n",
      "50913it [17:23, 52.13it/s]\u001b[A\n",
      "50945it [17:24, 52.16it/s]\u001b[A\n",
      "50977it [17:25, 52.15it/s]\u001b[A\n",
      "51009it [17:25, 51.65it/s]\u001b[A\n",
      "51041it [17:26, 51.70it/s]\u001b[A\n",
      "51073it [17:26, 51.85it/s]\u001b[A\n",
      "51105it [17:27, 51.95it/s]\u001b[A\n",
      "51137it [17:28, 52.00it/s]\u001b[A\n",
      "51169it [17:28, 52.02it/s]\u001b[A\n",
      "51201it [17:29, 52.12it/s]\u001b[A\n",
      "51233it [17:29, 51.98it/s]\u001b[A\n",
      "51265it [17:30, 52.03it/s]\u001b[A\n",
      "51297it [17:31, 52.04it/s]\u001b[A\n",
      "51329it [17:31, 51.99it/s]\u001b[A\n",
      "51361it [17:32, 51.88it/s]\u001b[A\n",
      "51393it [17:33, 52.06it/s]\u001b[A\n",
      "51425it [17:33, 52.14it/s]\u001b[A\n",
      "51457it [17:34, 52.19it/s]\u001b[A\n",
      "51489it [17:34, 52.18it/s]\u001b[A\n",
      "51521it [17:35, 52.09it/s]\u001b[A\n",
      "51553it [17:36, 52.16it/s]\u001b[A\n",
      "51585it [17:36, 52.12it/s]\u001b[A\n",
      "51617it [17:37, 52.10it/s]\u001b[A\n",
      "51649it [17:37, 52.15it/s]\u001b[A\n",
      "51681it [17:38, 52.13it/s]\u001b[A\n",
      "51713it [17:39, 52.16it/s]\u001b[A\n",
      "51745it [17:39, 52.13it/s]\u001b[A\n",
      "51777it [17:40, 52.14it/s]\u001b[A\n",
      "51809it [17:41, 52.08it/s]\u001b[A\n",
      "51841it [17:41, 52.10it/s]\u001b[A\n",
      "51873it [17:42, 52.12it/s]\u001b[A\n",
      "51905it [17:42, 52.06it/s]\u001b[A\n",
      "51937it [17:43, 52.07it/s]\u001b[A\n",
      "51969it [17:44, 52.10it/s]\u001b[A\n",
      "52001it [17:44, 52.24it/s]\u001b[A\n",
      "52033it [17:45, 52.07it/s]\u001b[A\n",
      "52065it [17:45, 52.08it/s]\u001b[A\n",
      "52097it [17:46, 52.19it/s]\u001b[A\n",
      "52129it [17:47, 52.08it/s]\u001b[A\n",
      "52161it [17:47, 52.07it/s]\u001b[A\n",
      "52193it [17:48, 52.06it/s]\u001b[A\n",
      "52225it [17:49, 52.06it/s]\u001b[A\n",
      "52257it [17:49, 52.03it/s]\u001b[A\n",
      "52289it [17:50, 52.00it/s]\u001b[A\n",
      "52321it [17:50, 52.11it/s]\u001b[A\n",
      "52353it [17:51, 52.10it/s]\u001b[A\n",
      "52385it [17:52, 52.11it/s]\u001b[A\n",
      "52417it [17:52, 52.20it/s]\u001b[A\n",
      "52449it [17:53, 52.13it/s]\u001b[A\n",
      "52481it [17:53, 52.16it/s]\u001b[A\n",
      "52513it [17:54, 52.04it/s]\u001b[A\n",
      "52545it [17:55, 52.07it/s]\u001b[A\n",
      "52577it [17:55, 52.14it/s]\u001b[A\n",
      "52609it [17:56, 52.01it/s]\u001b[A\n",
      "52641it [17:57, 51.77it/s]\u001b[A\n",
      "52673it [17:57, 51.79it/s]\u001b[A\n",
      "52705it [17:58, 51.83it/s]\u001b[A\n",
      "52737it [17:58, 51.93it/s]\u001b[A\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "52769it [17:59, 52.03it/s]\u001b[A\n",
      "52801it [18:00, 52.07it/s]\u001b[A\n",
      "52833it [18:00, 52.09it/s]\u001b[A\n",
      "52865it [18:01, 52.10it/s]\u001b[A\n",
      "52897it [18:01, 52.11it/s]\u001b[A\n",
      "52929it [18:02, 52.09it/s]\u001b[A\n",
      "52961it [18:03, 52.15it/s]\u001b[A\n",
      "52993it [18:03, 52.14it/s]\u001b[A\n",
      "53025it [18:04, 52.10it/s]\u001b[A\n",
      "53057it [18:04, 52.08it/s]\u001b[A\n",
      "53089it [18:05, 52.11it/s]\u001b[A\n",
      "53121it [18:06, 52.13it/s]\u001b[A\n",
      "53153it [18:06, 52.09it/s]\u001b[A\n",
      "53185it [18:07, 52.10it/s]\u001b[A\n",
      "53217it [18:08, 52.12it/s]\u001b[A\n",
      "53249it [18:08, 52.12it/s]\u001b[A\n",
      "53281it [18:09, 52.13it/s]\u001b[A\n",
      "53313it [18:09, 52.11it/s]\u001b[A\n",
      "53345it [18:10, 52.12it/s]\u001b[A\n",
      "53377it [18:11, 52.14it/s]\u001b[A\n",
      "53409it [18:11, 52.15it/s]\u001b[A\n",
      "53441it [18:12, 52.11it/s]\u001b[A\n",
      "53473it [18:12, 52.07it/s]\u001b[A\n",
      "53505it [18:13, 52.14it/s]\u001b[A\n",
      "53537it [18:14, 52.10it/s]\u001b[A\n",
      "53569it [18:14, 52.11it/s]\u001b[A\n",
      "53601it [18:15, 52.06it/s]\u001b[A\n",
      "53633it [18:16, 52.13it/s]\u001b[A\n",
      "53665it [18:16, 52.12it/s]\u001b[A\n",
      "53697it [18:17, 52.12it/s]\u001b[A\n",
      "53729it [18:17, 52.12it/s]\u001b[A\n",
      "53761it [18:18, 52.05it/s]\u001b[A\n",
      "53793it [18:19, 52.13it/s]\u001b[A\n",
      "53825it [18:19, 52.12it/s]\u001b[A\n",
      "53857it [18:20, 52.14it/s]\u001b[A\n",
      "53889it [18:20, 52.16it/s]\u001b[A\n",
      "53921it [18:21, 51.99it/s]\u001b[A\n",
      "53953it [18:22, 52.08it/s]\u001b[A\n",
      "53985it [18:22, 52.13it/s]\u001b[A\n",
      "54017it [18:23, 52.13it/s]\u001b[A\n",
      "54049it [18:24, 52.15it/s]\u001b[A\n",
      "54081it [18:24, 52.16it/s]\u001b[A\n",
      "54113it [18:25, 52.13it/s]\u001b[A\n",
      "54145it [18:25, 52.16it/s]\u001b[A\n",
      "54177it [18:26, 52.15it/s]\u001b[A\n",
      "54209it [18:27, 52.15it/s]\u001b[A\n",
      "54241it [18:27, 52.04it/s]\u001b[A\n",
      "54273it [18:28, 52.07it/s]\u001b[A\n",
      "54305it [18:28, 52.13it/s]\u001b[A\n",
      "54337it [18:29, 52.08it/s]\u001b[A\n",
      "54369it [18:30, 52.08it/s]\u001b[A\n",
      "54401it [18:30, 52.14it/s]\u001b[A\n",
      "54433it [18:31, 52.11it/s]\u001b[A\n",
      "54465it [18:32, 52.10it/s]\u001b[A\n",
      "54497it [18:32, 52.14it/s]\u001b[A\n",
      "54529it [18:33, 52.10it/s]\u001b[A\n",
      "54561it [18:33, 52.05it/s]\u001b[A\n",
      "54593it [18:34, 52.13it/s]\u001b[A\n",
      "54625it [18:35, 52.15it/s]\u001b[A\n",
      "54657it [18:35, 52.20it/s]\u001b[A\n",
      "54689it [18:36, 52.09it/s]\u001b[A\n",
      "54721it [18:36, 52.16it/s]\u001b[A\n",
      "54753it [18:37, 52.09it/s]\u001b[A\n",
      "54785it [18:38, 52.12it/s]\u001b[A\n",
      "54817it [18:38, 52.05it/s]\u001b[A\n",
      "54849it [18:39, 52.07it/s]\u001b[A\n",
      "54881it [18:39, 52.05it/s]\u001b[A\n",
      "54913it [18:40, 52.15it/s]\u001b[A\n",
      "54945it [18:41, 52.16it/s]\u001b[A\n",
      "54977it [18:41, 52.14it/s]\u001b[A\n",
      "55009it [18:42, 52.19it/s]\u001b[A\n",
      "55041it [18:43, 52.17it/s]\u001b[A\n",
      "55073it [18:43, 51.98it/s]\u001b[A\n",
      "55105it [18:44, 52.13it/s]\u001b[A\n",
      "55137it [18:44, 52.13it/s]\u001b[A\n",
      "55169it [18:45, 52.09it/s]\u001b[A\n",
      "55201it [18:46, 52.18it/s]\u001b[A\n",
      "55233it [18:46, 52.18it/s]\u001b[A\n",
      "55265it [18:47, 52.09it/s]\u001b[A\n",
      "55297it [18:47, 52.14it/s]\u001b[A\n",
      "55329it [18:48, 52.00it/s]\u001b[A\n",
      "55361it [18:49, 52.18it/s]\u001b[A\n",
      "55393it [18:49, 52.16it/s]\u001b[A\n",
      "55425it [18:50, 52.14it/s]\u001b[A\n",
      "55457it [18:51, 52.09it/s]\u001b[A\n",
      "55489it [18:51, 52.09it/s]\u001b[A\n",
      "55521it [18:52, 52.02it/s]\u001b[A\n",
      "55553it [18:52, 52.15it/s]\u001b[A\n",
      "55585it [18:53, 52.13it/s]\u001b[A\n",
      "55617it [18:54, 52.13it/s]\u001b[A\n",
      "55649it [18:54, 52.07it/s]\u001b[A\n",
      "55681it [18:55, 52.08it/s]\u001b[A\n",
      "55713it [18:55, 52.13it/s]\u001b[A\n",
      "55745it [18:56, 52.07it/s]\u001b[A\n",
      "55777it [18:57, 52.10it/s]\u001b[A\n",
      "55809it [18:57, 52.02it/s]\u001b[A\n",
      "55841it [18:58, 52.12it/s]\u001b[A\n",
      "55873it [18:59, 52.16it/s]\u001b[A\n",
      "55905it [18:59, 52.17it/s]\u001b[A\n",
      "55937it [19:00, 52.07it/s]\u001b[A\n",
      "55969it [19:00, 52.30it/s]\u001b[A\n",
      "56001it [19:01, 52.06it/s]\u001b[A\n",
      "56033it [19:02, 52.09it/s]\u001b[A\n",
      "56065it [19:02, 52.00it/s]\u001b[A\n",
      "56097it [19:03, 52.07it/s]\u001b[A\n",
      "56129it [19:03, 52.15it/s]\u001b[A\n",
      "56161it [19:04, 51.97it/s]\u001b[A\n",
      "56193it [19:05, 52.16it/s]\u001b[A\n",
      "56225it [19:05, 52.19it/s]\u001b[A\n",
      "56257it [19:06, 51.93it/s]\u001b[A\n",
      "56289it [19:07, 52.10it/s]\u001b[A\n",
      "56321it [19:07, 52.17it/s]\u001b[A\n",
      "56353it [19:08, 52.07it/s]\u001b[A\n",
      "56385it [19:08, 52.07it/s]\u001b[A\n",
      "56417it [19:09, 52.12it/s]\u001b[A\n",
      "56449it [19:10, 52.09it/s]\u001b[A\n",
      "56481it [19:10, 52.15it/s]\u001b[A\n",
      "56513it [19:11, 52.16it/s]\u001b[A\n",
      "56545it [19:11, 52.15it/s]\u001b[A\n",
      "56577it [19:12, 52.19it/s]\u001b[A\n",
      "56609it [19:13, 52.04it/s]\u001b[A\n",
      "56641it [19:13, 52.07it/s]\u001b[A\n",
      "56673it [19:14, 52.09it/s]\u001b[A\n",
      "56705it [19:14, 52.16it/s]\u001b[A\n",
      "56737it [19:15, 52.15it/s]\u001b[A\n",
      "56769it [19:16, 52.03it/s]\u001b[A\n",
      "56801it [19:16, 52.03it/s]\u001b[A\n",
      "56833it [19:17, 52.10it/s]\u001b[A\n",
      "56865it [19:18, 52.14it/s]\u001b[A\n",
      "56897it [19:18, 51.95it/s]\u001b[A\n",
      "56929it [19:19, 52.05it/s]\u001b[A\n",
      "56961it [19:19, 52.12it/s]\u001b[A\n",
      "56993it [19:20, 52.12it/s]\u001b[A\n",
      "57025it [19:21, 52.07it/s]\u001b[A\n",
      "57057it [19:21, 52.11it/s]\u001b[A\n",
      "57089it [19:22, 52.07it/s]\u001b[A\n",
      "57121it [19:22, 52.14it/s]\u001b[A\n",
      "57153it [19:23, 52.13it/s]\u001b[A\n",
      "57185it [19:24, 52.06it/s]\u001b[A\n",
      "57217it [19:24, 52.08it/s]\u001b[A\n",
      "57249it [19:25, 52.08it/s]\u001b[A\n",
      "57281it [19:26, 52.09it/s]\u001b[A\n",
      "57313it [19:26, 52.18it/s]\u001b[A\n",
      "57345it [19:27, 52.13it/s]\u001b[A\n",
      "57377it [19:27, 52.07it/s]\u001b[A\n",
      "57409it [19:28, 52.13it/s]\u001b[A\n",
      "57441it [19:29, 52.09it/s]\u001b[A\n",
      "57473it [19:29, 52.15it/s]\u001b[A\n",
      "57505it [19:30, 52.01it/s]\u001b[A\n",
      "57537it [19:30, 52.11it/s]\u001b[A\n",
      "57569it [19:31, 52.15it/s]\u001b[A\n",
      "57601it [19:32, 52.19it/s]\u001b[A\n",
      "57633it [19:32, 52.17it/s]\u001b[A\n",
      "57665it [19:33, 52.16it/s]\u001b[A\n",
      "57697it [19:34, 52.13it/s]\u001b[A\n",
      "57729it [19:34, 52.15it/s]\u001b[A\n",
      "57761it [19:35, 52.11it/s]\u001b[A\n",
      "57793it [19:35, 52.15it/s]\u001b[A\n",
      "57825it [19:36, 52.12it/s]\u001b[A\n",
      "57857it [19:37, 52.07it/s]\u001b[A\n",
      "57889it [19:37, 52.12it/s]\u001b[A\n",
      "57921it [19:38, 52.15it/s]\u001b[A\n",
      "57953it [19:38, 52.12it/s]\u001b[A\n",
      "57985it [19:39, 52.13it/s]\u001b[A\n",
      "58017it [19:40, 52.15it/s]\u001b[A\n",
      "58049it [19:40, 52.07it/s]\u001b[A\n",
      "58081it [19:41, 52.09it/s]\u001b[A\n",
      "58113it [19:42, 52.08it/s]\u001b[A\n",
      "58145it [19:42, 52.13it/s]\u001b[A\n",
      "58177it [19:43, 52.17it/s]\u001b[A\n",
      "58209it [19:43, 52.14it/s]\u001b[A\n",
      "58241it [19:44, 52.08it/s]\u001b[A\n",
      "58273it [19:45, 52.06it/s]\u001b[A\n",
      "58305it [19:45, 52.11it/s]\u001b[A\n",
      "58337it [19:46, 52.13it/s]\u001b[A\n",
      "58369it [19:46, 52.14it/s]\u001b[A\n",
      "58401it [19:47, 52.16it/s]\u001b[A\n",
      "58433it [19:48, 52.16it/s]\u001b[A\n",
      "58465it [19:48, 52.11it/s]\u001b[A\n",
      "58497it [19:49, 52.23it/s]\u001b[A\n",
      "58529it [19:49, 52.08it/s]\u001b[A\n",
      "58561it [19:50, 52.06it/s]\u001b[A\n",
      "58593it [19:51, 52.05it/s]\u001b[A\n",
      "58625it [19:51, 52.05it/s]\u001b[A\n",
      "58657it [19:52, 52.10it/s]\u001b[A\n",
      "58689it [19:53, 52.09it/s]\u001b[A\n",
      "58721it [19:53, 52.12it/s]\u001b[A\n",
      "58753it [19:54, 52.07it/s]\u001b[A\n",
      "58785it [19:54, 52.04it/s]\u001b[A\n",
      "58817it [19:55, 52.10it/s]\u001b[A\n",
      "58849it [19:56, 52.12it/s]\u001b[A\n",
      "58881it [19:56, 52.11it/s]\u001b[A\n",
      "58913it [19:57, 52.10it/s]\u001b[A\n",
      "58945it [19:57, 52.09it/s]\u001b[A\n",
      "58977it [19:58, 52.23it/s]\u001b[A\n",
      "59009it [19:59, 52.07it/s]\u001b[A\n",
      "59041it [19:59, 52.06it/s]\u001b[A\n",
      "59073it [20:00, 52.14it/s]\u001b[A\n",
      "59105it [20:01, 52.06it/s]\u001b[A\n",
      "59137it [20:01, 52.04it/s]\u001b[A\n",
      "59169it [20:02, 52.08it/s]\u001b[A\n",
      "59201it [20:02, 52.13it/s]\u001b[A\n",
      "59233it [20:03, 52.15it/s]\u001b[A\n",
      "59265it [20:04, 52.15it/s]\u001b[A\n",
      "59297it [20:04, 52.12it/s]\u001b[A\n",
      "59329it [20:05, 52.17it/s]\u001b[A\n",
      "59361it [20:05, 52.11it/s]\u001b[A\n",
      "59393it [20:06, 52.13it/s]\u001b[A\n",
      "59425it [20:07, 52.15it/s]\u001b[A\n",
      "59457it [20:07, 52.05it/s]\u001b[A\n",
      "59489it [20:08, 52.09it/s]\u001b[A\n",
      "59521it [20:09, 52.11it/s]\u001b[A\n",
      "59553it [20:09, 52.05it/s]\u001b[A\n",
      "59585it [20:10, 52.11it/s]\u001b[A\n",
      "59617it [20:10, 52.08it/s]\u001b[A\n",
      "59649it [20:11, 52.10it/s]\u001b[A\n",
      "59681it [20:12, 52.06it/s]\u001b[A\n",
      "59713it [20:12, 52.17it/s]\u001b[A\n",
      "59745it [20:13, 52.13it/s]\u001b[A\n",
      "59777it [20:13, 52.00it/s]\u001b[A\n",
      "59809it [20:14, 52.12it/s]\u001b[A\n",
      "59841it [20:15, 52.09it/s]\u001b[A\n",
      "59873it [20:15, 52.15it/s]\u001b[A\n",
      "59905it [20:16, 52.12it/s]\u001b[A\n",
      "59937it [20:17, 52.11it/s]\u001b[A\n",
      "59969it [20:17, 52.10it/s]\u001b[A\n",
      "60001it [20:18, 52.12it/s]\u001b[A\n",
      "60033it [20:18, 52.12it/s]\u001b[A\n",
      "60065it [20:19, 52.11it/s]\u001b[A\n",
      "60097it [20:20, 52.08it/s]\u001b[A\n",
      "60129it [20:20, 52.13it/s]\u001b[A\n",
      "60161it [20:21, 52.15it/s]\u001b[A\n",
      "60193it [20:21, 52.17it/s]\u001b[A\n",
      "60225it [20:22, 52.16it/s]\u001b[A\n",
      "60257it [20:23, 52.13it/s]\u001b[A\n",
      "60289it [20:23, 52.12it/s]\u001b[A\n",
      "60321it [20:24, 52.13it/s]\u001b[A\n",
      "60353it [20:24, 52.13it/s]\u001b[A\n",
      "60385it [20:25, 52.23it/s]\u001b[A\n",
      "60417it [20:26, 51.88it/s]\u001b[A\n",
      "60449it [20:26, 51.74it/s]\u001b[A\n",
      "60481it [20:27, 51.65it/s]\u001b[A\n",
      "60513it [20:28, 51.83it/s]\u001b[A\n",
      "60545it [20:28, 51.96it/s]\u001b[A\n",
      "60577it [20:29, 52.01it/s]\u001b[A\n",
      "60609it [20:29, 52.08it/s]\u001b[A\n",
      "60641it [20:30, 52.14it/s]\u001b[A\n",
      "60673it [20:31, 52.05it/s]\u001b[A\n",
      "60705it [20:31, 52.03it/s]\u001b[A\n",
      "60737it [20:32, 52.05it/s]\u001b[A\n",
      "60769it [20:32, 52.15it/s]\u001b[A\n",
      "60801it [20:33, 52.14it/s]\u001b[A\n",
      "60833it [20:34, 52.12it/s]\u001b[A\n",
      "60865it [20:34, 52.14it/s]\u001b[A\n",
      "60897it [20:35, 52.09it/s]\u001b[A\n",
      "60929it [20:36, 52.11it/s]\u001b[A\n",
      "60961it [20:36, 52.08it/s]\u001b[A\n",
      "60993it [20:37, 52.07it/s]\u001b[A\n",
      "61025it [20:37, 52.14it/s]\u001b[A\n",
      "61057it [20:38, 52.12it/s]\u001b[A\n",
      "61089it [20:39, 52.10it/s]\u001b[A\n",
      "61121it [20:39, 52.09it/s]\u001b[A\n",
      "61153it [20:40, 52.19it/s]\u001b[A\n",
      "61185it [20:40, 52.12it/s]\u001b[A\n",
      "61217it [20:41, 52.11it/s]\u001b[A\n",
      "61249it [20:42, 52.06it/s]\u001b[A\n",
      "61281it [20:42, 52.12it/s]\u001b[A\n",
      "61313it [20:43, 52.08it/s]\u001b[A\n",
      "61345it [20:44, 52.10it/s]\u001b[A\n",
      "61377it [20:44, 52.10it/s]\u001b[A\n",
      "61409it [20:45, 52.08it/s]\u001b[A\n",
      "61441it [20:45, 52.09it/s]\u001b[A\n",
      "61473it [20:46, 52.04it/s]\u001b[A\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "61505it [20:47, 52.10it/s]\u001b[A\n",
      "61537it [20:47, 52.12it/s]\u001b[A\n",
      "61569it [20:48, 52.11it/s]\u001b[A\n",
      "61601it [20:48, 52.12it/s]\u001b[A\n",
      "61633it [20:49, 52.04it/s]\u001b[A\n",
      "61665it [20:50, 52.14it/s]\u001b[A\n",
      "61697it [20:50, 52.20it/s]\u001b[A\n",
      "61729it [20:51, 52.17it/s]\u001b[A\n",
      "61761it [20:52, 52.15it/s]\u001b[A\n",
      "61793it [20:52, 52.09it/s]\u001b[A\n",
      "61825it [20:53, 52.13it/s]\u001b[A\n",
      "61857it [20:53, 52.15it/s]\u001b[A\n",
      "61889it [20:54, 52.15it/s]\u001b[A\n",
      "61921it [20:55, 52.10it/s]\u001b[A\n",
      "61953it [20:55, 52.10it/s]\u001b[A\n",
      "61985it [20:56, 52.11it/s]\u001b[A\n",
      "62017it [20:56, 52.11it/s]\u001b[A\n",
      "62049it [20:57, 52.09it/s]\u001b[A\n",
      "62081it [20:58, 52.07it/s]\u001b[A\n",
      "62113it [20:58, 52.12it/s]\u001b[A\n",
      "62145it [20:59, 52.02it/s]\u001b[A\n",
      "62177it [21:00, 51.67it/s]\u001b[A\n",
      "62209it [21:00, 51.77it/s]\u001b[A\n",
      "62241it [21:01, 51.91it/s]\u001b[A\n",
      "62273it [21:01, 52.00it/s]\u001b[A\n",
      "62305it [21:02, 52.01it/s]\u001b[A\n",
      "62337it [21:03, 52.05it/s]\u001b[A\n",
      "62369it [21:03, 51.99it/s]\u001b[A\n",
      "62401it [21:04, 52.10it/s]\u001b[A\n",
      "62433it [21:04, 52.09it/s]\u001b[A\n",
      "62465it [21:05, 52.18it/s]\u001b[A\n",
      "62497it [21:06, 51.89it/s]\u001b[A\n",
      "62529it [21:06, 51.82it/s]\u001b[A\n",
      "62561it [21:07, 51.77it/s]\u001b[A\n",
      "62593it [21:08, 51.89it/s]\u001b[A\n",
      "62625it [21:08, 51.92it/s]\u001b[A\n",
      "62657it [21:09, 51.94it/s]\u001b[A\n",
      "62689it [21:09, 51.98it/s]\u001b[A\n",
      "62721it [21:10, 52.03it/s]\u001b[A\n",
      "62753it [21:11, 52.02it/s]\u001b[A\n",
      "62785it [21:11, 52.08it/s]\u001b[A\n",
      "62817it [21:12, 52.03it/s]\u001b[A\n",
      "62849it [21:12, 52.07it/s]\u001b[A\n",
      "62881it [21:13, 52.09it/s]\u001b[A\n",
      "62913it [21:14, 52.15it/s]\u001b[A\n",
      "62945it [21:14, 52.00it/s]\u001b[A\n",
      "62977it [21:15, 52.07it/s]\u001b[A\n",
      "63009it [21:16, 52.05it/s]\u001b[A\n",
      "63041it [21:16, 52.09it/s]\u001b[A\n",
      "63073it [21:17, 52.13it/s]\u001b[A\n",
      "63105it [21:17, 52.12it/s]\u001b[A\n",
      "63137it [21:18, 52.07it/s]\u001b[A\n",
      "63169it [21:19, 52.04it/s]\u001b[A\n",
      "63201it [21:19, 52.09it/s]\u001b[A\n",
      "63233it [21:20, 52.09it/s]\u001b[A\n",
      "63265it [21:20, 52.06it/s]\u001b[A\n",
      "63297it [21:21, 52.09it/s]\u001b[A\n",
      "63329it [21:22, 51.99it/s]\u001b[A\n",
      "63361it [21:22, 52.10it/s]\u001b[A\n",
      "63393it [21:23, 52.08it/s]\u001b[A\n",
      "63425it [21:24, 52.11it/s]\u001b[A\n",
      "63457it [21:24, 52.15it/s]\u001b[A\n",
      "63489it [21:25, 52.15it/s]\u001b[A\n",
      "63521it [21:25, 52.16it/s]\u001b[A\n",
      "63553it [21:26, 52.09it/s]\u001b[A\n",
      "63585it [21:27, 52.10it/s]\u001b[A\n",
      "63617it [21:27, 52.36it/s]\u001b[A\n",
      "63649it [21:28, 52.37it/s]\u001b[A\n",
      "63681it [21:28, 52.56it/s]\u001b[A\n",
      "63713it [21:29, 52.55it/s]\u001b[A\n",
      "63745it [21:30, 52.25it/s]\u001b[A\n",
      "63777it [21:30, 52.20it/s]\u001b[A\n",
      "63809it [21:31, 52.18it/s]\u001b[A\n",
      "63841it [21:31, 52.11it/s]\u001b[A\n",
      "63873it [21:32, 52.07it/s]\u001b[A\n",
      "63905it [21:33, 52.05it/s]\u001b[A\n",
      "63937it [21:33, 52.11it/s]\u001b[A\n",
      "63969it [21:34, 52.09it/s]\u001b[A\n",
      "64001it [21:35, 52.12it/s]\u001b[A\n",
      "64033it [21:35, 52.30it/s]\u001b[A\n",
      "64065it [21:36, 52.09it/s]\u001b[A\n",
      "64097it [21:36, 52.11it/s]\u001b[A\n",
      "64129it [21:37, 52.03it/s]\u001b[A\n",
      "64161it [21:38, 52.10it/s]\u001b[A\n",
      "64193it [21:38, 52.05it/s]\u001b[A\n",
      "64225it [21:39, 52.09it/s]\u001b[A\n",
      "64257it [21:39, 52.12it/s]\u001b[A\n",
      "64289it [21:40, 52.14it/s]\u001b[A\n",
      "64321it [21:41, 52.05it/s]\u001b[A\n",
      "64353it [21:41, 52.09it/s]\u001b[A\n",
      "64385it [21:42, 52.07it/s]\u001b[A\n",
      "64417it [21:43, 52.13it/s]\u001b[A\n",
      "64449it [21:43, 52.15it/s]\u001b[A\n",
      "64481it [21:44, 52.12it/s]\u001b[A\n",
      "64513it [21:44, 52.08it/s]\u001b[A\n",
      "64545it [21:45, 52.02it/s]\u001b[A\n",
      "64577it [21:46, 52.16it/s]\u001b[A\n",
      "64609it [21:46, 52.12it/s]\u001b[A\n",
      "64641it [21:47, 52.11it/s]\u001b[A\n",
      "64673it [21:47, 52.14it/s]\u001b[A\n",
      "64705it [21:48, 52.01it/s]\u001b[A\n",
      "64737it [21:49, 52.11it/s]\u001b[A\n",
      "64769it [21:49, 52.09it/s]\u001b[A\n",
      "64801it [21:50, 52.04it/s]\u001b[A\n",
      "64833it [21:51, 52.09it/s]\u001b[A\n",
      "64865it [21:51, 52.08it/s]\u001b[A\n",
      "64897it [21:52, 52.13it/s]\u001b[A\n",
      "64929it [21:52, 52.13it/s]\u001b[A\n",
      "64961it [21:53, 52.14it/s]\u001b[A\n",
      "64993it [21:54, 52.15it/s]\u001b[A\n",
      "65025it [21:54, 52.06it/s]\u001b[A\n",
      "65057it [21:55, 52.08it/s]\u001b[A\n",
      "65089it [21:55, 52.16it/s]\u001b[A\n",
      "65121it [21:56, 52.15it/s]\u001b[A\n",
      "65153it [21:57, 52.13it/s]\u001b[A\n",
      "65185it [21:57, 51.96it/s]\u001b[A\n",
      "65217it [21:58, 52.14it/s]\u001b[A\n",
      "65249it [21:59, 52.08it/s]\u001b[A\n",
      "65281it [21:59, 52.12it/s]\u001b[A\n",
      "65313it [22:00, 52.08it/s]\u001b[A\n",
      "65345it [22:00, 52.15it/s]\u001b[A\n",
      "65377it [22:01, 52.03it/s]\u001b[A\n",
      "65409it [22:02, 52.09it/s]\u001b[A\n",
      "65441it [22:02, 52.15it/s]\u001b[A\n",
      "65473it [22:03, 52.15it/s]\u001b[A\n",
      "65505it [22:03, 52.21it/s]\u001b[A\n",
      "65537it [22:04, 52.11it/s]\u001b[A\n",
      "65569it [22:05, 52.09it/s]\u001b[A\n",
      "65601it [22:05, 52.13it/s]\u001b[A\n",
      "65633it [22:06, 52.10it/s]\u001b[A\n",
      "65665it [22:06, 52.11it/s]\u001b[A\n",
      "65697it [22:07, 52.18it/s]\u001b[A\n",
      "65729it [22:08, 52.08it/s]\u001b[A\n",
      "65761it [22:08, 52.14it/s]\u001b[A\n",
      "65793it [22:09, 52.15it/s]\u001b[A\n",
      "65825it [22:10, 52.11it/s]\u001b[A\n",
      "65857it [22:10, 52.08it/s]\u001b[A\n",
      "65889it [22:11, 52.15it/s]\u001b[A\n",
      "65921it [22:11, 52.03it/s]\u001b[A\n",
      "65953it [22:12, 52.10it/s]\u001b[A\n",
      "65985it [22:13, 52.14it/s]\u001b[A\n",
      "66017it [22:13, 52.15it/s]\u001b[A\n",
      "66049it [22:14, 52.08it/s]\u001b[A\n",
      "66081it [22:14, 52.14it/s]\u001b[A\n",
      "66113it [22:15, 52.05it/s]\u001b[A\n",
      "66145it [22:16, 52.00it/s]\u001b[A\n",
      "66177it [22:16, 52.12it/s]\u001b[A\n",
      "66209it [22:17, 52.12it/s]\u001b[A\n",
      "66241it [22:18, 52.16it/s]\u001b[A\n",
      "66273it [22:18, 52.10it/s]\u001b[A\n",
      "66305it [22:19, 52.11it/s]\u001b[A\n",
      "66337it [22:19, 52.14it/s]\u001b[A\n",
      "66369it [22:20, 52.11it/s]\u001b[A\n",
      "66401it [22:21, 52.15it/s]\u001b[A\n",
      "66433it [22:21, 52.18it/s]\u001b[A\n",
      "66465it [22:22, 52.08it/s]\u001b[A\n",
      "66497it [22:22, 52.09it/s]\u001b[A\n",
      "66529it [22:23, 51.97it/s]\u001b[A\n",
      "66561it [22:24, 52.07it/s]\u001b[A\n",
      "66593it [22:24, 52.16it/s]\u001b[A\n",
      "66625it [22:25, 52.04it/s]\u001b[A\n",
      "66657it [22:26, 52.10it/s]\u001b[A\n",
      "66689it [22:26, 52.17it/s]\u001b[A\n",
      "66721it [22:27, 52.14it/s]\u001b[A\n",
      "66753it [22:27, 52.10it/s]\u001b[A\n",
      "66785it [22:28, 52.14it/s]\u001b[A\n",
      "66817it [22:29, 52.09it/s]\u001b[A\n",
      "66849it [22:29, 52.12it/s]\u001b[A\n",
      "66881it [22:30, 52.07it/s]\u001b[A\n",
      "66913it [22:30, 52.09it/s]\u001b[A\n",
      "66945it [22:31, 51.99it/s]\u001b[A\n",
      "66977it [22:32, 52.15it/s]\u001b[A\n",
      "67009it [22:32, 52.14it/s]\u001b[A\n",
      "67041it [22:33, 52.14it/s]\u001b[A\n",
      "67073it [22:34, 52.16it/s]\u001b[A\n",
      "67105it [22:34, 52.14it/s]\u001b[A\n",
      "67137it [22:35, 52.12it/s]\u001b[A\n",
      "67169it [22:35, 52.08it/s]\u001b[A\n",
      "67201it [22:36, 52.16it/s]\u001b[A\n",
      "67233it [22:37, 52.21it/s]\u001b[A\n",
      "67265it [22:37, 52.11it/s]\u001b[A\n",
      "67297it [22:38, 52.10it/s]\u001b[A\n",
      "67329it [22:38, 52.14it/s]\u001b[A\n",
      "67361it [22:39, 52.07it/s]\u001b[A\n",
      "67393it [22:40, 52.05it/s]\u001b[A\n",
      "67425it [22:40, 52.12it/s]\u001b[A\n",
      "67457it [22:41, 52.13it/s]\u001b[A\n",
      "67489it [22:41, 52.08it/s]\u001b[A\n",
      "67521it [22:42, 52.10it/s]\u001b[A\n",
      "67553it [22:43, 52.11it/s]\u001b[A\n",
      "67585it [22:43, 52.07it/s]\u001b[A\n",
      "67617it [22:44, 52.09it/s]\u001b[A\n",
      "67649it [22:45, 52.12it/s]\u001b[A\n",
      "67681it [22:45, 52.11it/s]\u001b[A\n",
      "67713it [22:46, 52.07it/s]\u001b[A\n",
      "67745it [22:46, 52.15it/s]\u001b[A\n",
      "67777it [22:47, 52.10it/s]\u001b[A\n",
      "67809it [22:48, 52.06it/s]\u001b[A\n",
      "67841it [22:48, 52.20it/s]\u001b[A\n",
      "67873it [22:49, 52.09it/s]\u001b[A\n",
      "67905it [22:49, 52.07it/s]\u001b[A\n",
      "67937it [22:50, 52.07it/s]\u001b[A\n",
      "67969it [22:51, 52.05it/s]\u001b[A\n",
      "68001it [22:51, 52.05it/s]\u001b[A\n",
      "68033it [22:52, 52.10it/s]\u001b[A\n",
      "68065it [22:53, 52.09it/s]\u001b[A\n",
      "68097it [22:53, 52.14it/s]\u001b[A\n",
      "68129it [22:54, 52.14it/s]\u001b[A\n",
      "68161it [22:54, 52.10it/s]\u001b[A\n",
      "68193it [22:55, 52.15it/s]\u001b[A\n",
      "68225it [22:56, 52.08it/s]\u001b[A\n",
      "68257it [22:56, 52.13it/s]\u001b[A\n",
      "68289it [22:57, 52.01it/s]\u001b[A\n",
      "68321it [22:57, 52.20it/s]\u001b[A\n",
      "68353it [22:58, 52.14it/s]\u001b[A\n",
      "68385it [22:59, 52.17it/s]\u001b[A\n",
      "68417it [22:59, 52.16it/s]\u001b[A\n",
      "68449it [23:00, 52.16it/s]\u001b[A\n",
      "68481it [23:01, 52.16it/s]\u001b[A\n",
      "68513it [23:01, 52.15it/s]\u001b[A\n",
      "68545it [23:02, 52.20it/s]\u001b[A\n",
      "68577it [23:02, 52.06it/s]\u001b[A\n",
      "68609it [23:03, 52.08it/s]\u001b[A\n",
      "68641it [23:04, 52.24it/s]\u001b[A\n",
      "68673it [23:04, 52.15it/s]\u001b[A\n",
      "68705it [23:05, 51.99it/s]\u001b[A\n",
      "68737it [23:05, 52.05it/s]\u001b[A\n",
      "68769it [23:06, 52.13it/s]\u001b[A\n",
      "68801it [23:07, 52.10it/s]\u001b[A\n",
      "68833it [23:07, 52.17it/s]\u001b[A\n",
      "68865it [23:08, 52.06it/s]\u001b[A\n",
      "68897it [23:09, 52.08it/s]\u001b[A\n",
      "68929it [23:09, 52.13it/s]\u001b[A\n",
      "68961it [23:10, 52.10it/s]\u001b[A\n",
      "68993it [23:10, 52.09it/s]\u001b[A\n",
      "69025it [23:11, 52.05it/s]\u001b[A\n",
      "69057it [23:12, 52.00it/s]\u001b[A\n",
      "69089it [23:12, 52.11it/s]\u001b[A\n",
      "69121it [23:13, 52.14it/s]\u001b[A\n",
      "69153it [23:13, 52.10it/s]\u001b[A\n",
      "69185it [23:14, 52.13it/s]\u001b[A\n",
      "69217it [23:15, 52.02it/s]\u001b[A\n",
      "69249it [23:15, 52.05it/s]\u001b[A\n",
      "69281it [23:16, 52.01it/s]\u001b[A\n",
      "69313it [23:16, 52.13it/s]\u001b[A\n",
      "69345it [23:17, 52.10it/s]\u001b[A\n",
      "69377it [23:18, 52.09it/s]\u001b[A\n",
      "69409it [23:18, 52.12it/s]\u001b[A\n",
      "69441it [23:19, 52.12it/s]\u001b[A\n",
      "69473it [23:20, 51.70it/s]\u001b[A\n",
      "69505it [23:20, 51.70it/s]\u001b[A\n",
      "69537it [23:21, 51.87it/s]\u001b[A\n",
      "69569it [23:21, 51.90it/s]\u001b[A\n",
      "69601it [23:22, 51.94it/s]\u001b[A\n",
      "69633it [23:23, 52.00it/s]\u001b[A\n",
      "69665it [23:23, 51.97it/s]\u001b[A\n",
      "69697it [23:24, 52.04it/s]\u001b[A\n",
      "69729it [23:24, 52.13it/s]\u001b[A\n",
      "69761it [23:25, 51.97it/s]\u001b[A\n",
      "69793it [23:26, 52.06it/s]\u001b[A\n",
      "69825it [23:26, 52.10it/s]\u001b[A\n",
      "69857it [23:27, 52.09it/s]\u001b[A\n",
      "69889it [23:28, 52.16it/s]\u001b[A\n",
      "69921it [23:28, 52.16it/s]\u001b[A\n",
      "69953it [23:29, 52.14it/s]\u001b[A\n",
      "69985it [23:29, 52.14it/s]\u001b[A\n",
      "70017it [23:30, 52.07it/s]\u001b[A\n",
      "70049it [23:31, 52.11it/s]\u001b[A\n",
      "70081it [23:31, 52.14it/s]\u001b[A\n",
      "70113it [23:32, 52.10it/s]\u001b[A\n",
      "70145it [23:32, 52.14it/s]\u001b[A\n",
      "70177it [23:33, 52.10it/s]\u001b[A\n",
      "70209it [23:34, 52.10it/s]\u001b[A\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "70241it [23:34, 51.89it/s]\u001b[A\n",
      "70273it [23:35, 51.84it/s]\u001b[A\n",
      "70305it [23:36, 51.91it/s]\u001b[A\n",
      "70337it [23:36, 51.78it/s]\u001b[A\n",
      "70369it [23:37, 51.87it/s]\u001b[A\n",
      "70401it [23:37, 51.89it/s]\u001b[A\n",
      "70433it [23:38, 52.01it/s]\u001b[A\n",
      "70465it [23:39, 52.04it/s]\u001b[A\n",
      "70497it [23:39, 51.96it/s]\u001b[A\n",
      "70529it [23:40, 52.03it/s]\u001b[A\n",
      "70561it [23:40, 52.04it/s]\u001b[A\n",
      "70593it [23:41, 52.09it/s]\u001b[A\n",
      "70625it [23:42, 52.11it/s]\u001b[A\n",
      "70657it [23:42, 52.10it/s]\u001b[A\n",
      "70689it [23:43, 52.09it/s]\u001b[A\n",
      "70721it [23:44, 52.13it/s]\u001b[A\n",
      "70753it [23:44, 52.16it/s]\u001b[A\n",
      "70785it [23:45, 52.06it/s]\u001b[A\n",
      "70817it [23:45, 52.05it/s]\u001b[A\n",
      "70849it [23:46, 52.12it/s]\u001b[A\n",
      "70881it [23:47, 52.14it/s]\u001b[A\n",
      "70913it [23:47, 52.13it/s]\u001b[A\n",
      "70945it [23:48, 52.10it/s]\u001b[A\n",
      "70977it [23:48, 52.13it/s]\u001b[A\n",
      "71009it [23:49, 52.09it/s]\u001b[A\n",
      "71041it [23:50, 52.09it/s]\u001b[A\n",
      "71073it [23:50, 52.07it/s]\u001b[A\n",
      "71105it [23:51, 52.12it/s]\u001b[A\n",
      "71137it [23:52, 52.08it/s]\u001b[A\n",
      "71169it [23:52, 52.11it/s]\u001b[A\n",
      "71201it [23:53, 52.09it/s]\u001b[A\n",
      "71233it [23:53, 52.26it/s]\u001b[A\n",
      "71265it [23:54, 52.07it/s]\u001b[A\n",
      "71297it [23:55, 51.99it/s]\u001b[A\n",
      "71329it [23:55, 52.04it/s]\u001b[A\n",
      "71361it [23:56, 52.07it/s]\u001b[A\n",
      "71393it [23:56, 52.15it/s]\u001b[A\n",
      "71425it [23:57, 52.10it/s]\u001b[A\n",
      "71457it [23:58, 52.10it/s]\u001b[A\n",
      "71489it [23:58, 52.08it/s]\u001b[A\n",
      "71521it [23:59, 52.08it/s]\u001b[A\n",
      "71553it [24:00, 52.12it/s]\u001b[A\n",
      "71585it [24:00, 52.07it/s]\u001b[A\n",
      "71617it [24:01, 52.12it/s]\u001b[A\n",
      "71649it [24:01, 52.10it/s]\u001b[A\n",
      "71681it [24:02, 52.10it/s]\u001b[A\n",
      "71713it [24:03, 52.14it/s]\u001b[A\n",
      "71745it [24:03, 52.09it/s]\u001b[A\n",
      "71777it [24:04, 52.15it/s]\u001b[A\n",
      "71809it [24:04, 52.14it/s]\u001b[A\n",
      "71841it [24:05, 52.15it/s]\u001b[A\n",
      "71873it [24:06, 52.08it/s]\u001b[A\n",
      "71905it [24:06, 52.11it/s]\u001b[A\n",
      "71937it [24:07, 52.11it/s]\u001b[A\n",
      "71969it [24:08, 52.19it/s]\u001b[A\n",
      "72001it [24:08, 52.09it/s]\u001b[A\n",
      "72033it [24:09, 52.10it/s]\u001b[A\n",
      "72065it [24:09, 52.06it/s]\u001b[A\n",
      "72097it [24:10, 52.10it/s]\u001b[A\n",
      "72129it [24:11, 52.09it/s]\u001b[A\n",
      "72161it [24:11, 52.10it/s]\u001b[A\n",
      "72193it [24:12, 52.09it/s]\u001b[A\n",
      "72225it [24:12, 52.11it/s]\u001b[A\n",
      "72257it [24:13, 52.10it/s]\u001b[A\n",
      "72289it [24:14, 52.14it/s]\u001b[A\n",
      "72321it [24:14, 52.04it/s]\u001b[A\n",
      "72353it [24:15, 52.07it/s]\u001b[A\n",
      "72385it [24:15, 52.14it/s]\u001b[A\n",
      "72417it [24:16, 52.15it/s]\u001b[A\n",
      "72449it [24:17, 52.17it/s]\u001b[A\n",
      "72481it [24:17, 51.98it/s]\u001b[A\n",
      "72513it [24:18, 51.82it/s]\u001b[A\n",
      "72545it [24:19, 51.91it/s]\u001b[A\n",
      "72577it [24:19, 51.84it/s]\u001b[A\n",
      "72609it [24:20, 51.89it/s]\u001b[A\n",
      "72641it [24:20, 51.95it/s]\u001b[A\n",
      "72673it [24:21, 52.04it/s]\u001b[A\n",
      "72705it [24:22, 52.01it/s]\u001b[A\n",
      "72737it [24:22, 52.05it/s]\u001b[A\n",
      "72769it [24:23, 52.06it/s]\u001b[A\n",
      "72801it [24:23, 52.11it/s]\u001b[A\n",
      "72833it [24:24, 52.08it/s]\u001b[A\n",
      "72865it [24:25, 52.12it/s]\u001b[A\n",
      "72897it [24:25, 52.09it/s]\u001b[A\n",
      "72929it [24:26, 52.12it/s]\u001b[A\n",
      "72961it [24:27, 52.11it/s]\u001b[A\n",
      "72993it [24:27, 52.10it/s]\u001b[A\n",
      "73025it [24:28, 52.10it/s]\u001b[A\n",
      "73057it [24:28, 52.13it/s]\u001b[A\n",
      "73089it [24:29, 52.16it/s]\u001b[A\n",
      "73121it [24:30, 51.95it/s]\u001b[A\n",
      "73153it [24:30, 52.09it/s]\u001b[A\n",
      "73185it [24:31, 52.10it/s]\u001b[A\n",
      "73217it [24:31, 52.10it/s]\u001b[A\n",
      "73249it [24:32, 52.13it/s]\u001b[A\n",
      "73281it [24:33, 52.12it/s]\u001b[A\n",
      "73313it [24:33, 52.10it/s]\u001b[A\n",
      "73345it [24:34, 52.08it/s]\u001b[A\n",
      "73377it [24:35, 52.10it/s]\u001b[A\n",
      "73409it [24:35, 52.10it/s]\u001b[A\n",
      "73441it [24:36, 52.12it/s]\u001b[A\n",
      "73473it [24:36, 52.15it/s]\u001b[A\n",
      "73505it [24:37, 52.11it/s]\u001b[A\n",
      "73537it [24:38, 52.10it/s]\u001b[A\n",
      "73569it [24:38, 52.12it/s]\u001b[A\n",
      "73601it [24:39, 51.99it/s]\u001b[A\n",
      "73633it [24:39, 52.09it/s]\u001b[A\n",
      "73665it [24:40, 52.08it/s]\u001b[A\n",
      "73697it [24:41, 52.10it/s]\u001b[A\n",
      "73729it [24:41, 52.23it/s]\u001b[A\n",
      "73761it [24:42, 52.09it/s]\u001b[A\n",
      "73793it [24:43, 52.08it/s]\u001b[A\n",
      "73825it [24:43, 52.12it/s]\u001b[A\n",
      "73857it [24:44, 52.14it/s]\u001b[A\n",
      "73889it [24:44, 52.15it/s]\u001b[A\n",
      "73921it [24:45, 52.07it/s]\u001b[A\n",
      "73953it [24:46, 52.11it/s]\u001b[A\n",
      "73985it [24:46, 52.25it/s]\u001b[A\n",
      "74017it [24:47, 52.10it/s]\u001b[A\n",
      "74049it [24:47, 52.07it/s]\u001b[A\n",
      "74081it [24:48, 52.14it/s]\u001b[A\n",
      "74113it [24:49, 52.11it/s]\u001b[A\n",
      "74145it [24:49, 52.17it/s]\u001b[A\n",
      "74177it [24:50, 52.02it/s]\u001b[A\n",
      "74209it [24:51, 52.13it/s]\u001b[A\n",
      "74241it [24:51, 52.18it/s]\u001b[A\n",
      "74273it [24:52, 52.11it/s]\u001b[A\n",
      "74305it [24:52, 52.08it/s]\u001b[A\n",
      "74337it [24:53, 52.13it/s]\u001b[A\n",
      "74369it [24:54, 52.16it/s]\u001b[A\n",
      "74401it [24:54, 52.09it/s]\u001b[A\n",
      "74433it [24:55, 52.12it/s]\u001b[A\n",
      "74465it [24:55, 52.08it/s]\u001b[A\n",
      "74497it [24:56, 52.10it/s]\u001b[A\n",
      "74529it [24:57, 52.24it/s]\u001b[A\n",
      "74561it [24:57, 52.07it/s]\u001b[A\n",
      "74593it [24:58, 52.14it/s]\u001b[A\n",
      "74625it [24:58, 52.05it/s]\u001b[A\n",
      "74657it [24:59, 52.12it/s]\u001b[A\n",
      "74689it [25:00, 52.11it/s]\u001b[A\n",
      "74721it [25:00, 52.11it/s]\u001b[A\n",
      "74753it [25:01, 52.09it/s]\u001b[A\n",
      "74785it [25:02, 52.03it/s]\u001b[A\n",
      "74817it [25:02, 51.99it/s]\u001b[A\n",
      "74849it [25:03, 52.15it/s]\u001b[A\n",
      "74881it [25:03, 52.07it/s]\u001b[A\n",
      "74913it [25:04, 52.14it/s]\u001b[A\n",
      "74945it [25:05, 52.15it/s]\u001b[A\n",
      "74977it [25:05, 52.03it/s]\u001b[A\n",
      "75009it [25:06, 52.11it/s]\u001b[A\n",
      "75041it [25:06, 52.11it/s]\u001b[A\n",
      "75073it [25:07, 52.15it/s]\u001b[A\n",
      "75105it [25:08, 52.08it/s]\u001b[A\n",
      "75137it [25:08, 52.11it/s]\u001b[A\n",
      "75169it [25:09, 52.09it/s]\u001b[A\n",
      "75201it [25:10, 52.11it/s]\u001b[A\n",
      "75233it [25:10, 52.12it/s]\u001b[A\n",
      "75265it [25:11, 51.92it/s]\u001b[A\n",
      "75297it [25:11, 52.14it/s]\u001b[A\n",
      "75329it [25:12, 52.08it/s]\u001b[A\n",
      "75361it [25:13, 52.16it/s]\u001b[A\n",
      "75393it [25:13, 52.12it/s]\u001b[A\n",
      "75425it [25:14, 52.13it/s]\u001b[A\n",
      "75457it [25:14, 52.04it/s]\u001b[A\n",
      "75489it [25:15, 52.09it/s]\u001b[A\n",
      "75521it [25:16, 52.00it/s]\u001b[A\n",
      "75553it [25:16, 51.69it/s]\u001b[A\n",
      "75585it [25:17, 51.77it/s]\u001b[A\n",
      "75617it [25:18, 51.94it/s]\u001b[A\n",
      "75649it [25:18, 51.97it/s]\u001b[A\n",
      "75681it [25:19, 52.00it/s]\u001b[A\n",
      "75713it [25:19, 52.07it/s]\u001b[A\n",
      "75745it [25:20, 52.01it/s]\u001b[A\n",
      "75777it [25:21, 52.05it/s]\u001b[A\n",
      "75809it [25:21, 52.11it/s]\u001b[A\n",
      "75841it [25:22, 52.10it/s]\u001b[A\n",
      "75873it [25:22, 52.13it/s]\u001b[A\n",
      "75905it [25:23, 52.11it/s]\u001b[A\n",
      "75937it [25:24, 52.06it/s]\u001b[A\n",
      "75969it [25:24, 52.11it/s]\u001b[A\n",
      "76001it [25:25, 52.12it/s]\u001b[A\n",
      "76033it [25:26, 52.18it/s]\u001b[A\n",
      "76065it [25:26, 51.92it/s]\u001b[A\n",
      "76097it [25:27, 52.18it/s]\u001b[A\n",
      "76129it [25:27, 52.12it/s]\u001b[A\n",
      "76161it [25:28, 52.17it/s]\u001b[A\n",
      "76193it [25:29, 52.07it/s]\u001b[A\n",
      "76225it [25:29, 52.12it/s]\u001b[A\n",
      "76257it [25:30, 52.10it/s]\u001b[A\n",
      "76289it [25:30, 52.08it/s]\u001b[A\n",
      "76321it [25:31, 52.05it/s]\u001b[A\n",
      "76353it [25:32, 52.11it/s]\u001b[A\n",
      "76385it [25:32, 52.16it/s]\u001b[A\n",
      "76417it [25:33, 52.21it/s]\u001b[A\n",
      "76449it [25:34, 52.14it/s]\u001b[A\n",
      "76481it [25:34, 52.15it/s]\u001b[A\n",
      "76513it [25:35, 52.11it/s]\u001b[A\n",
      "76545it [25:35, 52.15it/s]\u001b[A\n",
      "76577it [25:36, 52.18it/s]\u001b[A\n",
      "76609it [25:37, 52.10it/s]\u001b[A\n",
      "76641it [25:37, 52.11it/s]\u001b[A\n",
      "76673it [25:38, 52.11it/s]\u001b[A\n",
      "76705it [25:38, 52.14it/s]\u001b[A\n",
      "76737it [25:39, 52.17it/s]\u001b[A\n",
      "76769it [25:40, 52.11it/s]\u001b[A\n",
      "76801it [25:40, 52.11it/s]\u001b[A\n",
      "76833it [25:41, 52.05it/s]\u001b[A\n",
      "76865it [25:41, 52.17it/s]\u001b[A\n",
      "76897it [25:42, 52.11it/s]\u001b[A\n",
      "76929it [25:43, 52.10it/s]\u001b[A\n",
      "76961it [25:43, 52.03it/s]\u001b[A\n",
      "76993it [25:44, 52.10it/s]\u001b[A\n",
      "77025it [25:45, 52.13it/s]\u001b[A\n",
      "77057it [25:45, 52.11it/s]\u001b[A\n",
      "77089it [25:46, 52.10it/s]\u001b[A\n",
      "77121it [25:46, 52.10it/s]\u001b[A\n",
      "77153it [25:47, 52.13it/s]\u001b[A\n",
      "77185it [25:48, 52.13it/s]\u001b[A\n",
      "77217it [25:48, 52.17it/s]\u001b[A\n",
      "77249it [25:49, 52.05it/s]\u001b[A\n",
      "77281it [25:49, 52.09it/s]\u001b[A\n",
      "77313it [25:50, 52.10it/s]\u001b[A\n",
      "77345it [25:51, 52.09it/s]\u001b[A\n",
      "77377it [25:51, 52.12it/s]\u001b[A\n",
      "77409it [25:52, 52.09it/s]\u001b[A\n",
      "77441it [25:53, 52.18it/s]\u001b[A\n",
      "77473it [25:53, 52.13it/s]\u001b[A\n",
      "77505it [25:54, 52.08it/s]\u001b[A\n",
      "77537it [25:54, 52.15it/s]\u001b[A\n",
      "77569it [25:55, 52.08it/s]\u001b[A\n",
      "77601it [25:56, 52.10it/s]\u001b[A\n",
      "77633it [25:56, 52.06it/s]\u001b[A\n",
      "77665it [25:57, 52.13it/s]\u001b[A\n",
      "77697it [25:57, 52.13it/s]\u001b[A\n",
      "77729it [25:58, 52.11it/s]\u001b[A\n",
      "77761it [25:59, 52.09it/s]\u001b[A\n",
      "77793it [25:59, 52.15it/s]\u001b[A\n",
      "77825it [26:00, 52.13it/s]\u001b[A\n",
      "77857it [26:01, 51.58it/s]\u001b[A\n",
      "77889it [26:01, 51.67it/s]\u001b[A\n",
      "77921it [26:02, 51.86it/s]\u001b[A\n",
      "77953it [26:02, 51.93it/s]\u001b[A\n",
      "77985it [26:03, 52.04it/s]\u001b[A\n",
      "78017it [26:04, 52.02it/s]\u001b[A\n",
      "78049it [26:04, 52.04it/s]\u001b[A\n",
      "78081it [26:05, 51.99it/s]\u001b[A\n",
      "78113it [26:05, 52.02it/s]\u001b[A\n",
      "78145it [26:06, 52.07it/s]\u001b[A\n",
      "78177it [26:07, 52.14it/s]\u001b[A\n",
      "78209it [26:07, 52.09it/s]\u001b[A\n",
      "78241it [26:08, 52.07it/s]\u001b[A\n",
      "78273it [26:09, 52.11it/s]\u001b[A\n",
      "78305it [26:09, 52.08it/s]\u001b[A\n",
      "78337it [26:10, 52.15it/s]\u001b[A\n",
      "78369it [26:10, 52.03it/s]\u001b[A\n",
      "78401it [26:11, 52.14it/s]\u001b[A\n",
      "78433it [26:12, 52.10it/s]\u001b[A\n",
      "78465it [26:12, 52.09it/s]\u001b[A\n",
      "78497it [26:13, 52.11it/s]\u001b[A\n",
      "78529it [26:13, 52.05it/s]\u001b[A\n",
      "78561it [26:14, 52.03it/s]\u001b[A\n",
      "78593it [26:15, 52.11it/s]\u001b[A\n",
      "78625it [26:15, 52.06it/s]\u001b[A\n",
      "78657it [26:16, 52.11it/s]\u001b[A\n",
      "78689it [26:17, 52.10it/s]\u001b[A\n",
      "78721it [26:17, 52.07it/s]\u001b[A\n",
      "78753it [26:18, 52.12it/s]\u001b[A\n",
      "78785it [26:18, 52.14it/s]\u001b[A\n",
      "78817it [26:19, 52.06it/s]\u001b[A\n",
      "78849it [26:20, 52.15it/s]\u001b[A\n",
      "78881it [26:20, 52.31it/s]\u001b[A\n",
      "78913it [26:21, 52.08it/s]\u001b[A\n",
      "78945it [26:21, 51.94it/s]\u001b[A\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "78977it [26:22, 52.11it/s]\u001b[A\n",
      "79009it [26:23, 52.14it/s]\u001b[A\n",
      "79041it [26:23, 52.13it/s]\u001b[A\n",
      "79073it [26:24, 52.12it/s]\u001b[A\n",
      "79105it [26:25, 52.16it/s]\u001b[A\n",
      "79137it [26:25, 52.10it/s]\u001b[A\n",
      "79169it [26:26, 52.09it/s]\u001b[A\n",
      "79201it [26:26, 52.05it/s]\u001b[A\n",
      "79233it [26:27, 52.06it/s]\u001b[A\n",
      "79265it [26:28, 52.15it/s]\u001b[A\n",
      "79297it [26:28, 52.04it/s]\u001b[A\n",
      "79329it [26:29, 52.13it/s]\u001b[A\n",
      "79361it [26:29, 52.17it/s]\u001b[A\n",
      "79393it [26:30, 52.04it/s]\u001b[A\n",
      "79425it [26:31, 52.04it/s]\u001b[A\n",
      "79457it [26:31, 52.05it/s]\u001b[A\n",
      "79489it [26:32, 52.13it/s]\u001b[A\n",
      "79521it [26:32, 52.13it/s]\u001b[A\n",
      "79553it [26:33, 52.16it/s]\u001b[A\n",
      "79585it [26:34, 52.14it/s]\u001b[A\n",
      "79617it [26:34, 52.12it/s]\u001b[A\n",
      "79649it [26:35, 52.16it/s]\u001b[A\n",
      "79681it [26:36, 52.13it/s]\u001b[A\n",
      "79713it [26:36, 52.09it/s]\u001b[A\n",
      "79745it [26:37, 52.13it/s]\u001b[A\n",
      "79777it [26:37, 52.12it/s]\u001b[A\n",
      "79809it [26:38, 52.05it/s]\u001b[A\n",
      "79841it [26:39, 52.06it/s]\u001b[A\n",
      "79873it [26:39, 52.12it/s]\u001b[A\n",
      "79905it [26:40, 52.11it/s]\u001b[A\n",
      "79937it [26:40, 52.12it/s]\u001b[A\n",
      "79969it [26:41, 52.15it/s]\u001b[A\n",
      "80001it [26:42, 52.05it/s]\u001b[A\n",
      "80033it [26:42, 52.10it/s]\u001b[A\n",
      "80065it [26:43, 52.16it/s]\u001b[A\n",
      "80097it [26:44, 52.13it/s]\u001b[A\n",
      "80129it [26:44, 52.12it/s]\u001b[A\n",
      "80161it [26:45, 52.08it/s]\u001b[A\n",
      "80193it [26:45, 52.14it/s]\u001b[A\n",
      "80225it [26:46, 52.09it/s]\u001b[A\n",
      "80257it [26:47, 52.14it/s]\u001b[A\n",
      "80289it [26:47, 52.13it/s]\u001b[A\n",
      "80321it [26:48, 52.12it/s]\u001b[A\n",
      "80353it [26:48, 52.09it/s]\u001b[A\n",
      "80385it [26:49, 52.15it/s]\u001b[A\n",
      "80417it [26:50, 52.09it/s]\u001b[A\n",
      "80449it [26:50, 52.06it/s]\u001b[A\n",
      "80481it [26:51, 52.11it/s]\u001b[A\n",
      "80513it [26:52, 52.14it/s]\u001b[A\n",
      "80545it [26:52, 52.13it/s]\u001b[A\n",
      "80577it [26:53, 52.16it/s]\u001b[A\n",
      "80609it [26:53, 52.08it/s]\u001b[A\n",
      "80641it [26:54, 52.09it/s]\u001b[A\n",
      "80673it [26:55, 52.11it/s]\u001b[A\n",
      "80705it [26:55, 52.00it/s]\u001b[A\n",
      "80737it [26:56, 52.14it/s]\u001b[A\n",
      "80769it [26:56, 52.17it/s]\u001b[A\n",
      "80801it [26:57, 52.12it/s]\u001b[A\n",
      "80833it [26:58, 52.16it/s]\u001b[A\n",
      "80865it [26:58, 52.14it/s]\u001b[A\n",
      "80897it [26:59, 51.94it/s]\u001b[A\n",
      "80929it [27:00, 52.20it/s]\u001b[A\n",
      "80961it [27:00, 52.16it/s]\u001b[A\n",
      "80993it [27:01, 52.21it/s]\u001b[A\n",
      "81025it [27:01, 52.19it/s]\u001b[A\n",
      "81057it [27:02, 52.02it/s]\u001b[A\n",
      "81089it [27:03, 52.16it/s]\u001b[A\n",
      "81121it [27:03, 52.12it/s]\u001b[A\n",
      "81153it [27:04, 52.16it/s]\u001b[A\n",
      "81185it [27:04, 52.07it/s]\u001b[A\n",
      "81217it [27:05, 52.08it/s]\u001b[A\n",
      "81249it [27:06, 52.12it/s]\u001b[A\n",
      "81281it [27:06, 52.12it/s]\u001b[A\n",
      "81313it [27:07, 52.03it/s]\u001b[A\n",
      "81345it [27:07, 52.03it/s]\u001b[A\n",
      "81377it [27:08, 52.04it/s]\u001b[A\n",
      "81409it [27:09, 52.07it/s]\u001b[A\n",
      "81441it [27:09, 52.14it/s]\u001b[A\n",
      "81473it [27:10, 52.16it/s]\u001b[A\n",
      "81505it [27:11, 52.12it/s]\u001b[A\n",
      "81537it [27:11, 52.11it/s]\u001b[A\n",
      "81569it [27:12, 51.99it/s]\u001b[A\n",
      "81601it [27:12, 52.18it/s]\u001b[A\n",
      "81633it [27:13, 52.15it/s]\u001b[A\n",
      "81665it [27:14, 52.09it/s]\u001b[A\n",
      "81697it [27:14, 52.14it/s]\u001b[A\n",
      "81729it [27:15, 52.12it/s]\u001b[A\n",
      "81761it [27:15, 52.18it/s]\u001b[A\n",
      "81793it [27:16, 52.00it/s]\u001b[A\n",
      "81825it [27:17, 52.00it/s]\u001b[A\n",
      "81857it [27:17, 52.10it/s]\u001b[A\n",
      "81889it [27:18, 52.14it/s]\u001b[A\n",
      "81921it [27:19, 52.11it/s]\u001b[A\n",
      "81953it [27:19, 52.15it/s]\u001b[A\n",
      "81985it [27:20, 52.05it/s]\u001b[A\n",
      "82017it [27:20, 52.12it/s]\u001b[A\n",
      "82049it [27:21, 52.12it/s]\u001b[A\n",
      "82081it [27:22, 52.06it/s]\u001b[A\n",
      "82113it [27:22, 52.14it/s]\u001b[A\n",
      "82145it [27:23, 52.04it/s]\u001b[A\n",
      "82177it [27:23, 52.11it/s]\u001b[A\n",
      "82209it [27:24, 52.10it/s]\u001b[A\n",
      "82241it [27:25, 52.08it/s]\u001b[A\n",
      "82273it [27:25, 52.06it/s]\u001b[A\n",
      "82305it [27:26, 52.04it/s]\u001b[A\n",
      "82337it [27:27, 52.11it/s]\u001b[A\n",
      "82369it [27:27, 52.08it/s]\u001b[A\n",
      "82401it [27:28, 52.09it/s]\u001b[A\n",
      "82433it [27:28, 52.16it/s]\u001b[A\n",
      "82465it [27:29, 52.07it/s]\u001b[A\n",
      "82497it [27:30, 52.17it/s]\u001b[A\n",
      "82529it [27:30, 52.10it/s]\u001b[A\n",
      "82561it [27:31, 52.13it/s]\u001b[A\n",
      "82593it [27:31, 52.12it/s]\u001b[A\n",
      "82625it [27:32, 52.10it/s]\u001b[A\n",
      "82657it [27:33, 52.12it/s]\u001b[A\n",
      "82689it [27:33, 52.16it/s]\u001b[A\n",
      "82721it [27:34, 52.09it/s]\u001b[A\n",
      "82753it [27:35, 52.16it/s]\u001b[A\n",
      "82785it [27:35, 52.05it/s]\u001b[A\n",
      "82817it [27:36, 52.14it/s]\u001b[A\n",
      "82849it [27:36, 52.10it/s]\u001b[A\n",
      "82881it [27:37, 52.14it/s]\u001b[A\n",
      "82913it [27:38, 52.07it/s]\u001b[A\n",
      "82945it [27:38, 52.08it/s]\u001b[A\n",
      "82977it [27:39, 52.13it/s]\u001b[A\n",
      "83009it [27:39, 52.13it/s]\u001b[A\n",
      "83041it [27:40, 52.02it/s]\u001b[A\n",
      "83073it [27:41, 52.05it/s]\u001b[A\n",
      "83105it [27:41, 52.02it/s]\u001b[A\n",
      "83137it [27:42, 52.05it/s]\u001b[A\n",
      "83169it [27:43, 51.89it/s]\u001b[A\n",
      "83201it [27:43, 51.83it/s]\u001b[A\n",
      "83233it [27:44, 51.78it/s]\u001b[A\n",
      "83265it [27:44, 51.85it/s]\u001b[A\n",
      "83297it [27:45, 51.97it/s]\u001b[A\n",
      "83329it [27:46, 52.00it/s]\u001b[A\n",
      "83361it [27:46, 51.98it/s]\u001b[A\n",
      "83393it [27:47, 52.06it/s]\u001b[A\n",
      "83425it [27:47, 52.03it/s]\u001b[A\n",
      "83457it [27:48, 52.07it/s]\u001b[A\n",
      "83489it [27:49, 52.11it/s]\u001b[A\n",
      "83521it [27:49, 52.05it/s]\u001b[A\n",
      "83553it [27:50, 52.10it/s]\u001b[A\n",
      "83585it [27:51, 52.11it/s]\u001b[A\n",
      "83617it [27:51, 52.09it/s]\u001b[A\n",
      "83649it [27:52, 52.15it/s]\u001b[A\n",
      "83681it [27:52, 52.07it/s]\u001b[A\n",
      "83713it [27:53, 52.08it/s]\u001b[A\n",
      "83745it [27:54, 52.13it/s]\u001b[A\n",
      "83777it [27:54, 52.06it/s]\u001b[A\n",
      "83809it [27:55, 52.07it/s]\u001b[A\n",
      "83841it [27:55, 52.16it/s]\u001b[A\n",
      "83873it [27:56, 52.13it/s]\u001b[A\n",
      "83905it [27:57, 52.14it/s]\u001b[A\n",
      "83937it [27:57, 52.13it/s]\u001b[A\n",
      "83969it [27:58, 52.12it/s]\u001b[A\n",
      "84001it [27:58, 52.11it/s]\u001b[A\n",
      "84033it [27:59, 52.07it/s]\u001b[A\n",
      "84065it [28:00, 52.07it/s]\u001b[A\n",
      "84097it [28:00, 52.14it/s]\u001b[A\n",
      "84129it [28:01, 52.06it/s]\u001b[A\n",
      "84161it [28:02, 52.12it/s]\u001b[A\n",
      "84193it [28:02, 52.08it/s]\u001b[A\n",
      "84225it [28:03, 52.11it/s]\u001b[A\n",
      "84257it [28:03, 52.17it/s]\u001b[A\n",
      "84289it [28:04, 52.06it/s]\u001b[A\n",
      "84321it [28:05, 52.10it/s]\u001b[A\n",
      "84353it [28:05, 52.10it/s]\u001b[A\n",
      "84385it [28:06, 52.16it/s]\u001b[A\n",
      "84417it [28:06, 52.12it/s]\u001b[A\n",
      "84449it [28:07, 52.11it/s]\u001b[A\n",
      "84481it [28:08, 52.09it/s]\u001b[A\n",
      "84513it [28:08, 52.10it/s]\u001b[A\n",
      "84545it [28:09, 52.06it/s]\u001b[A\n",
      "84577it [28:10, 52.14it/s]\u001b[A\n",
      "84609it [28:10, 52.11it/s]\u001b[A\n",
      "84641it [28:11, 52.05it/s]\u001b[A\n",
      "84673it [28:11, 52.08it/s]\u001b[A\n",
      "84705it [28:12, 52.16it/s]\u001b[A\n",
      "84737it [28:13, 52.11it/s]\u001b[A\n",
      "84769it [28:13, 52.13it/s]\u001b[A\n",
      "84801it [28:14, 52.08it/s]\u001b[A\n",
      "84833it [28:14, 52.11it/s]\u001b[A\n",
      "84865it [28:15, 52.06it/s]\u001b[A\n",
      "84897it [28:16, 52.14it/s]\u001b[A\n",
      "84929it [28:16, 52.08it/s]\u001b[A\n",
      "84961it [28:17, 52.13it/s]\u001b[A\n",
      "84993it [28:18, 52.14it/s]\u001b[A\n",
      "85025it [28:18, 52.10it/s]\u001b[A\n",
      "85057it [28:19, 52.12it/s]\u001b[A\n",
      "85089it [28:19, 52.11it/s]\u001b[A\n",
      "85121it [28:20, 52.12it/s]\u001b[A\n",
      "85153it [28:21, 52.13it/s]\u001b[A\n",
      "85185it [28:21, 52.10it/s]\u001b[A\n",
      "85217it [28:22, 52.14it/s]\u001b[A\n",
      "85249it [28:22, 52.09it/s]\u001b[A\n",
      "85281it [28:23, 52.10it/s]\u001b[A\n",
      "85313it [28:24, 52.11it/s]\u001b[A\n",
      "85345it [28:24, 52.13it/s]\u001b[A\n",
      "85377it [28:25, 52.18it/s]\u001b[A\n",
      "85409it [28:26, 52.08it/s]\u001b[A\n",
      "85441it [28:26, 52.15it/s]\u001b[A\n",
      "85473it [28:27, 52.08it/s]\u001b[A\n",
      "85505it [28:27, 52.10it/s]\u001b[A\n",
      "85537it [28:28, 52.06it/s]\u001b[A\n",
      "85569it [28:29, 52.13it/s]\u001b[A\n",
      "85601it [28:29, 51.85it/s]\u001b[A\n",
      "85633it [28:30, 51.63it/s]\u001b[A\n",
      "85665it [28:30, 51.78it/s]\u001b[A\n",
      "85697it [28:31, 51.86it/s]\u001b[A\n",
      "85729it [28:32, 51.90it/s]\u001b[A\n",
      "85761it [28:32, 51.97it/s]\u001b[A\n",
      "85793it [28:33, 52.04it/s]\u001b[A\n",
      "85825it [28:34, 51.99it/s]\u001b[A\n",
      "85857it [28:34, 52.07it/s]\u001b[A\n",
      "85889it [28:35, 52.15it/s]\u001b[A\n",
      "85921it [28:35, 52.19it/s]\u001b[A\n",
      "85953it [28:36, 52.15it/s]\u001b[A\n",
      "85985it [28:37, 52.12it/s]\u001b[A\n",
      "86017it [28:37, 52.13it/s]\u001b[A\n",
      "86049it [28:38, 52.16it/s]\u001b[A\n",
      "86081it [28:38, 52.09it/s]\u001b[A\n",
      "86113it [28:39, 52.12it/s]\u001b[A\n",
      "86145it [28:40, 52.14it/s]\u001b[A\n",
      "86177it [28:40, 52.18it/s]\u001b[A\n",
      "86209it [28:41, 52.08it/s]\u001b[A\n",
      "86241it [28:42, 52.03it/s]\u001b[A\n",
      "86273it [28:42, 52.05it/s]\u001b[A\n",
      "86305it [28:43, 51.94it/s]\u001b[A\n",
      "86337it [28:43, 51.64it/s]\u001b[A\n",
      "86369it [28:44, 51.79it/s]\u001b[A\n",
      "86401it [28:45, 51.91it/s]\u001b[A\n",
      "86433it [28:45, 51.91it/s]\u001b[A\n",
      "86465it [28:46, 51.99it/s]\u001b[A\n",
      "86497it [28:46, 52.09it/s]\u001b[A\n",
      "86529it [28:47, 52.09it/s]\u001b[A\n",
      "86561it [28:48, 52.12it/s]\u001b[A\n",
      "86593it [28:48, 51.99it/s]\u001b[A\n",
      "86625it [28:49, 52.07it/s]\u001b[A\n",
      "86657it [28:50, 52.13it/s]\u001b[A\n",
      "86689it [28:50, 52.16it/s]\u001b[A\n",
      "86721it [28:51, 52.24it/s]\u001b[A\n",
      "86753it [28:51, 52.04it/s]\u001b[A\n",
      "86785it [28:52, 52.08it/s]\u001b[A\n",
      "86817it [28:53, 52.08it/s]\u001b[A\n",
      "86849it [28:53, 52.07it/s]\u001b[A\n",
      "86881it [28:54, 52.17it/s]\u001b[A\n",
      "86913it [28:54, 52.12it/s]\u001b[A\n",
      "86945it [28:55, 52.07it/s]\u001b[A\n",
      "86977it [28:56, 52.09it/s]\u001b[A\n",
      "87009it [28:56, 52.22it/s]\u001b[A\n",
      "87041it [28:57, 52.03it/s]\u001b[A\n",
      "87073it [28:57, 52.15it/s]\u001b[A\n",
      "87105it [28:58, 52.05it/s]\u001b[A\n",
      "87137it [28:59, 52.07it/s]\u001b[A\n",
      "87169it [28:59, 52.07it/s]\u001b[A\n",
      "87201it [29:00, 52.08it/s]\u001b[A\n",
      "87233it [29:01, 52.08it/s]\u001b[A\n",
      "87265it [29:01, 52.11it/s]\u001b[A\n",
      "87297it [29:02, 52.09it/s]\u001b[A\n",
      "87329it [29:02, 52.11it/s]\u001b[A\n",
      "87361it [29:03, 52.08it/s]\u001b[A\n",
      "87393it [29:04, 52.05it/s]\u001b[A\n",
      "87425it [29:04, 52.11it/s]\u001b[A\n",
      "87457it [29:05, 52.08it/s]\u001b[A\n",
      "87489it [29:05, 52.06it/s]\u001b[A\n",
      "87521it [29:06, 52.13it/s]\u001b[A\n",
      "87553it [29:07, 52.11it/s]\u001b[A\n",
      "87585it [29:07, 52.13it/s]\u001b[A\n",
      "87617it [29:08, 52.14it/s]\u001b[A\n",
      "87649it [29:09, 52.18it/s]\u001b[A\n",
      "87681it [29:09, 52.08it/s]\u001b[A\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "87713it [29:10, 52.10it/s]\u001b[A\n",
      "87745it [29:10, 52.11it/s]\u001b[A\n",
      "87777it [29:11, 52.12it/s]\u001b[A\n",
      "87809it [29:12, 52.17it/s]\u001b[A\n",
      "87841it [29:12, 52.06it/s]\u001b[A\n",
      "87873it [29:13, 52.14it/s]\u001b[A\n",
      "87905it [29:13, 52.08it/s]\u001b[A\n",
      "87937it [29:14, 51.99it/s]\u001b[A\n",
      "87969it [29:15, 52.18it/s]\u001b[A\n",
      "88001it [29:15, 52.12it/s]\u001b[A\n",
      "88033it [29:16, 52.00it/s]\u001b[A\n",
      "88065it [29:17, 52.12it/s]\u001b[A\n",
      "88097it [29:17, 52.16it/s]\u001b[A\n",
      "88129it [29:18, 52.16it/s]\u001b[A\n",
      "88161it [29:18, 52.12it/s]\u001b[A\n",
      "88193it [29:19, 52.16it/s]\u001b[A\n",
      "88225it [29:20, 52.10it/s]\u001b[A\n",
      "88257it [29:20, 52.14it/s]\u001b[A\n",
      "88289it [29:21, 51.99it/s]\u001b[A\n",
      "88321it [29:21, 52.14it/s]\u001b[A\n",
      "88353it [29:22, 52.10it/s]\u001b[A\n",
      "88385it [29:23, 52.12it/s]\u001b[A\n",
      "88417it [29:23, 52.09it/s]\u001b[A\n",
      "88449it [29:24, 52.19it/s]\u001b[A\n",
      "88481it [29:25, 52.16it/s]\u001b[A\n",
      "88513it [29:25, 52.07it/s]\u001b[A\n",
      "88545it [29:26, 52.16it/s]\u001b[A\n",
      "88577it [29:26, 52.12it/s]\u001b[A\n",
      "88609it [29:27, 52.16it/s]\u001b[A\n",
      "88641it [29:28, 52.14it/s]\u001b[A\n",
      "88673it [29:28, 52.18it/s]\u001b[A\n",
      "88705it [29:29, 52.13it/s]\u001b[A\n",
      "88737it [29:29, 52.13it/s]\u001b[A\n",
      "88769it [29:30, 52.14it/s]\u001b[A\n",
      "88801it [29:31, 52.09it/s]\u001b[A\n",
      "88833it [29:31, 52.13it/s]\u001b[A\n",
      "88865it [29:32, 52.06it/s]\u001b[A\n",
      "88897it [29:33, 51.84it/s]\u001b[A\n",
      "88929it [29:33, 51.69it/s]\u001b[A\n",
      "88961it [29:34, 51.85it/s]\u001b[A\n",
      "88993it [29:34, 51.91it/s]\u001b[A\n",
      "89025it [29:35, 51.94it/s]\u001b[A\n",
      "89057it [29:36, 51.81it/s]\u001b[A\n",
      "89089it [29:36, 52.14it/s]\u001b[A\n",
      "89121it [29:37, 52.11it/s]\u001b[A\n",
      "89153it [29:37, 52.15it/s]\u001b[A\n",
      "89185it [29:38, 52.10it/s]\u001b[A\n",
      "89217it [29:39, 51.67it/s]\u001b[A\n",
      "89249it [29:39, 51.37it/s]\u001b[A\n",
      "89281it [29:40, 51.46it/s]\u001b[A\n",
      "89313it [29:41, 51.63it/s]\u001b[A\n",
      "89345it [29:41, 51.77it/s]\u001b[A\n",
      "89377it [29:42, 51.88it/s]\u001b[A\n",
      "89409it [29:42, 51.89it/s]\u001b[A\n",
      "89441it [29:43, 52.04it/s]\u001b[A\n",
      "89473it [29:44, 52.07it/s]\u001b[A\n",
      "89505it [29:44, 52.00it/s]\u001b[A\n",
      "89537it [29:45, 52.03it/s]\u001b[A\n",
      "89569it [29:45, 52.00it/s]\u001b[A\n",
      "89601it [29:46, 52.10it/s]\u001b[A\n",
      "89633it [29:47, 52.15it/s]\u001b[A\n",
      "89665it [29:47, 52.08it/s]\u001b[A\n",
      "89697it [29:48, 52.10it/s]\u001b[A\n",
      "89729it [29:49, 52.12it/s]\u001b[A\n",
      "89761it [29:49, 52.14it/s]\u001b[A\n",
      "89793it [29:50, 52.18it/s]\u001b[A\n",
      "89825it [29:50, 52.13it/s]\u001b[A\n",
      "89857it [29:51, 52.11it/s]\u001b[A\n",
      "89889it [29:52, 52.14it/s]\u001b[A\n",
      "89921it [29:52, 52.07it/s]\u001b[A\n",
      "89953it [29:53, 52.13it/s]\u001b[A\n",
      "89985it [29:53, 52.10it/s]\u001b[A\n",
      "90017it [29:54, 52.04it/s]\u001b[A\n",
      "90049it [29:55, 52.09it/s]\u001b[A\n",
      "90081it [29:55, 51.95it/s]\u001b[A\n",
      "90113it [29:56, 52.10it/s]\u001b[A\n",
      "90145it [29:57, 52.12it/s]\u001b[A\n",
      "90177it [29:57, 52.07it/s]\u001b[A\n",
      "90209it [29:58, 52.12it/s]\u001b[A\n",
      "90241it [29:58, 52.09it/s]\u001b[A\n",
      "90273it [29:59, 52.19it/s]\u001b[A\n",
      "90305it [30:00, 52.13it/s]\u001b[A\n",
      "90337it [30:00, 52.11it/s]\u001b[A\n",
      "90369it [30:01, 52.10it/s]\u001b[A\n",
      "90401it [30:01, 52.08it/s]\u001b[A\n",
      "90433it [30:02, 52.00it/s]\u001b[A\n",
      "90465it [30:03, 52.06it/s]\u001b[A\n",
      "90497it [30:03, 52.02it/s]\u001b[A\n",
      "90529it [30:04, 52.03it/s]\u001b[A\n",
      "90561it [30:04, 52.10it/s]\u001b[A\n",
      "90593it [30:05, 52.16it/s]\u001b[A\n",
      "90625it [30:06, 52.07it/s]\u001b[A\n",
      "90657it [30:06, 51.97it/s]\u001b[A\n",
      "90689it [30:07, 52.13it/s]\u001b[A\n",
      "90721it [30:08, 52.08it/s]\u001b[A\n",
      "90753it [30:08, 52.23it/s]\u001b[A\n",
      "90785it [30:09, 52.14it/s]\u001b[A\n",
      "90817it [30:09, 52.05it/s]\u001b[A\n",
      "90849it [30:10, 51.89it/s]\u001b[A\n",
      "90881it [30:11, 51.66it/s]\u001b[A\n",
      "90913it [30:11, 51.69it/s]\u001b[A\n",
      "90945it [30:12, 51.92it/s]\u001b[A\n",
      "90977it [30:12, 51.97it/s]\u001b[A\n",
      "91009it [30:13, 52.05it/s]\u001b[A\n",
      "91041it [30:14, 52.06it/s]\u001b[A\n",
      "91073it [30:14, 52.06it/s]\u001b[A\n",
      "91105it [30:15, 52.06it/s]\u001b[A\n",
      "91137it [30:16, 52.12it/s]\u001b[A\n",
      "91169it [30:16, 52.04it/s]\u001b[A\n",
      "91201it [30:17, 52.09it/s]\u001b[A\n",
      "91233it [30:17, 52.09it/s]\u001b[A\n",
      "91265it [30:18, 52.13it/s]\u001b[A\n",
      "91297it [30:19, 52.10it/s]\u001b[A\n",
      "91329it [30:19, 52.05it/s]\u001b[A\n",
      "91361it [30:20, 51.92it/s]\u001b[A\n",
      "91393it [30:20, 52.17it/s]\u001b[A\n",
      "91425it [30:21, 52.12it/s]\u001b[A\n",
      "91457it [30:22, 52.11it/s]\u001b[A\n",
      "91489it [30:22, 52.04it/s]\u001b[A\n",
      "91521it [30:23, 52.14it/s]\u001b[A\n",
      "91553it [30:24, 52.13it/s]\u001b[A\n",
      "91585it [30:24, 52.11it/s]\u001b[A\n",
      "91617it [30:25, 52.12it/s]\u001b[A\n",
      "91649it [30:25, 52.02it/s]\u001b[A\n",
      "91681it [30:26, 52.16it/s]\u001b[A\n",
      "91713it [30:27, 52.08it/s]\u001b[A\n",
      "91745it [30:27, 52.10it/s]\u001b[A\n",
      "91777it [30:28, 52.10it/s]\u001b[A\n",
      "91809it [30:28, 52.00it/s]\u001b[A\n",
      "91841it [30:29, 52.04it/s]\u001b[A\n",
      "91873it [30:30, 52.16it/s]\u001b[A\n",
      "91905it [30:30, 52.12it/s]\u001b[A\n",
      "91937it [30:31, 52.14it/s]\u001b[A\n",
      "91969it [30:32, 52.09it/s]\u001b[A\n",
      "92001it [30:32, 52.12it/s]\u001b[A\n",
      "92033it [30:33, 52.11it/s]\u001b[A\n",
      "92065it [30:33, 52.17it/s]\u001b[A\n",
      "92097it [30:34, 52.08it/s]\u001b[A\n",
      "92129it [30:35, 52.09it/s]\u001b[A\n",
      "92161it [30:35, 52.10it/s]\u001b[A\n",
      "92193it [30:36, 52.08it/s]\u001b[A\n",
      "92225it [30:36, 52.13it/s]\u001b[A\n",
      "92257it [30:37, 52.12it/s]\u001b[A\n",
      "92289it [30:38, 51.94it/s]\u001b[A\n",
      "92321it [30:38, 52.14it/s]\u001b[A\n",
      "92353it [30:39, 52.18it/s]\u001b[A\n",
      "92385it [30:40, 52.14it/s]\u001b[A\n",
      "92417it [30:40, 52.13it/s]\u001b[A\n",
      "92449it [30:41, 52.13it/s]\u001b[A\n",
      "92481it [30:41, 52.11it/s]\u001b[A\n",
      "92513it [30:42, 52.09it/s]\u001b[A\n",
      "92545it [30:43, 52.16it/s]\u001b[A\n",
      "92577it [30:43, 52.10it/s]\u001b[A\n",
      "92609it [30:44, 52.08it/s]\u001b[A\n",
      "92641it [30:44, 52.09it/s]\u001b[A\n",
      "92673it [30:45, 52.11it/s]\u001b[A\n",
      "92705it [30:46, 52.05it/s]\u001b[A\n",
      "92737it [30:46, 52.10it/s]\u001b[A\n",
      "92769it [30:47, 52.11it/s]\u001b[A\n",
      "92801it [30:48, 52.06it/s]\u001b[A\n",
      "92833it [30:48, 52.11it/s]\u001b[A\n",
      "92865it [30:49, 52.10it/s]\u001b[A\n",
      "92897it [30:49, 52.13it/s]\u001b[A\n",
      "92929it [30:50, 52.12it/s]\u001b[A\n",
      "92961it [30:51, 52.09it/s]\u001b[A\n",
      "92993it [30:51, 52.13it/s]\u001b[A\n",
      "93025it [30:52, 52.24it/s]\u001b[A\n",
      "93057it [30:52, 52.07it/s]\u001b[A\n",
      "93089it [30:53, 52.09it/s]\u001b[A\n",
      "93121it [30:54, 52.11it/s]\u001b[A\n",
      "93153it [30:54, 52.09it/s]\u001b[A\n",
      "93185it [30:55, 52.12it/s]\u001b[A\n",
      "93217it [30:55, 52.15it/s]\u001b[A\n",
      "93249it [30:56, 52.13it/s]\u001b[A\n",
      "93281it [30:57, 52.09it/s]\u001b[A\n",
      "93313it [30:57, 52.14it/s]\u001b[A\n",
      "93345it [30:58, 52.12it/s]\u001b[A\n",
      "93377it [30:59, 52.18it/s]\u001b[A\n",
      "93409it [30:59, 52.10it/s]\u001b[A\n",
      "93441it [31:00, 52.05it/s]\u001b[A\n",
      "93473it [31:00, 52.11it/s]\u001b[A\n",
      "93505it [31:01, 52.10it/s]\u001b[A\n",
      "93537it [31:02, 52.08it/s]\u001b[A\n",
      "93569it [31:02, 52.11it/s]\u001b[A\n",
      "93601it [31:03, 52.12it/s]\u001b[A\n",
      "93633it [31:03, 52.15it/s]\u001b[A\n",
      "93665it [31:04, 52.14it/s]\u001b[A\n",
      "93697it [31:05, 52.16it/s]\u001b[A\n",
      "93729it [31:05, 52.08it/s]\u001b[A\n",
      "93761it [31:06, 52.15it/s]\u001b[A\n",
      "93793it [31:07, 51.86it/s]\u001b[A\n",
      "93825it [31:07, 52.16it/s]\u001b[A\n",
      "93857it [31:08, 52.15it/s]\u001b[A\n",
      "93889it [31:08, 52.16it/s]\u001b[A\n",
      "93921it [31:09, 52.07it/s]\u001b[A\n",
      "93953it [31:10, 51.96it/s]\u001b[A\n",
      "93985it [31:10, 51.78it/s]\u001b[A\n",
      "94017it [31:11, 51.84it/s]\u001b[A\n",
      "94049it [31:11, 51.96it/s]\u001b[A\n",
      "94081it [31:12, 52.01it/s]\u001b[A\n",
      "94113it [31:13, 51.99it/s]\u001b[A\n",
      "94145it [31:13, 52.06it/s]\u001b[A\n",
      "94177it [31:14, 52.06it/s]\u001b[A\n",
      "94209it [31:15, 52.07it/s]\u001b[A\n",
      "94241it [31:15, 52.08it/s]\u001b[A\n",
      "94273it [31:16, 52.04it/s]\u001b[A\n",
      "94305it [31:16, 52.07it/s]\u001b[A\n",
      "94337it [31:17, 52.11it/s]\u001b[A\n",
      "94369it [31:18, 52.06it/s]\u001b[A\n",
      "94401it [31:18, 52.11it/s]\u001b[A\n",
      "94433it [31:19, 52.06it/s]\u001b[A\n",
      "94465it [31:19, 52.13it/s]\u001b[A\n",
      "94497it [31:20, 52.13it/s]\u001b[A\n",
      "94529it [31:21, 52.11it/s]\u001b[A\n",
      "94561it [31:21, 52.11it/s]\u001b[A\n",
      "94593it [31:22, 52.17it/s]\u001b[A\n",
      "94625it [31:23, 52.09it/s]\u001b[A\n",
      "94657it [31:23, 52.14it/s]\u001b[A\n",
      "94689it [31:24, 52.13it/s]\u001b[A\n",
      "94721it [31:24, 52.11it/s]\u001b[A\n",
      "94753it [31:25, 52.19it/s]\u001b[A\n",
      "94785it [31:26, 52.13it/s]\u001b[A\n",
      "94817it [31:26, 52.20it/s]\u001b[A\n",
      "94849it [31:27, 52.16it/s]\u001b[A\n",
      "94881it [31:27, 52.12it/s]\u001b[A\n",
      "94913it [31:28, 52.14it/s]\u001b[A\n",
      "94945it [31:29, 52.11it/s]\u001b[A\n",
      "94977it [31:29, 52.08it/s]\u001b[A\n",
      "95009it [31:30, 51.94it/s]\u001b[A\n",
      "95041it [31:30, 52.19it/s]\u001b[A\n",
      "95073it [31:31, 52.17it/s]\u001b[A\n",
      "95105it [31:32, 52.20it/s]\u001b[A\n",
      "95137it [31:32, 52.19it/s]\u001b[A\n",
      "95169it [31:33, 52.04it/s]\u001b[A\n",
      "95201it [31:34, 52.13it/s]\u001b[A\n",
      "95233it [31:34, 52.22it/s]\u001b[A\n",
      "95265it [31:35, 52.04it/s]\u001b[A\n",
      "95297it [31:35, 52.28it/s]\u001b[A\n",
      "95329it [31:36, 52.12it/s]\u001b[A\n",
      "95361it [31:37, 52.07it/s]\u001b[A\n",
      "95393it [31:37, 52.03it/s]\u001b[A\n",
      "95425it [31:38, 51.94it/s]\u001b[A\n",
      "95457it [31:38, 52.10it/s]\u001b[A\n",
      "95489it [31:39, 52.11it/s]\u001b[A\n",
      "95521it [31:40, 52.11it/s]\u001b[A\n",
      "95553it [31:40, 52.13it/s]\u001b[A\n",
      "95585it [31:41, 52.14it/s]\u001b[A\n",
      "95617it [31:42, 52.03it/s]\u001b[A\n",
      "95649it [31:42, 52.09it/s]\u001b[A\n",
      "95681it [31:43, 52.12it/s]\u001b[A\n",
      "95713it [31:43, 52.15it/s]\u001b[A\n",
      "95745it [31:44, 52.11it/s]\u001b[A\n",
      "95777it [31:45, 51.99it/s]\u001b[A\n",
      "95809it [31:45, 52.01it/s]\u001b[A\n",
      "95841it [31:46, 52.10it/s]\u001b[A\n",
      "95873it [31:46, 52.12it/s]\u001b[A\n",
      "95905it [31:47, 52.15it/s]\u001b[A\n",
      "95937it [31:48, 52.09it/s]\u001b[A\n",
      "95969it [31:48, 52.08it/s]\u001b[A\n",
      "96001it [31:49, 52.13it/s]\u001b[A\n",
      "96033it [31:50, 52.16it/s]\u001b[A\n",
      "96065it [31:50, 52.13it/s]\u001b[A\n",
      "96097it [31:51, 52.02it/s]\u001b[A\n",
      "96129it [31:51, 52.20it/s]\u001b[A\n",
      "96161it [31:52, 52.23it/s]\u001b[A\n",
      "96193it [31:53, 52.12it/s]\u001b[A\n",
      "96225it [31:53, 52.14it/s]\u001b[A\n",
      "96257it [31:54, 52.08it/s]\u001b[A\n",
      "96289it [31:54, 52.16it/s]\u001b[A\n",
      "96321it [31:55, 52.17it/s]\u001b[A\n",
      "96353it [31:56, 52.15it/s]\u001b[A\n",
      "96385it [31:56, 52.18it/s]\u001b[A\n",
      "96417it [31:57, 52.18it/s]\u001b[A\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "96449it [31:58, 52.13it/s]\u001b[A\n",
      "96481it [31:58, 52.16it/s]\u001b[A\n",
      "96513it [31:59, 52.12it/s]\u001b[A\n",
      "96545it [31:59, 52.07it/s]\u001b[A\n",
      "96577it [32:00, 52.12it/s]\u001b[A\n",
      "96609it [32:01, 52.09it/s]\u001b[A\n",
      "96641it [32:01, 52.12it/s]\u001b[A\n",
      "96673it [32:02, 52.18it/s]\u001b[A\n",
      "96705it [32:02, 52.08it/s]\u001b[A\n",
      "96737it [32:03, 52.13it/s]\u001b[A\n",
      "96769it [32:04, 52.12it/s]\u001b[A\n",
      "96801it [32:04, 52.13it/s]\u001b[A\n",
      "96833it [32:05, 52.15it/s]\u001b[A\n",
      "96865it [32:05, 52.14it/s]\u001b[A\n",
      "96897it [32:06, 52.11it/s]\u001b[A\n",
      "96929it [32:07, 52.10it/s]\u001b[A\n",
      "96961it [32:07, 52.04it/s]\u001b[A\n",
      "96993it [32:08, 52.09it/s]\u001b[A\n",
      "97025it [32:09, 52.14it/s]\u001b[A\n",
      "97057it [32:09, 52.14it/s]\u001b[A\n",
      "97089it [32:10, 52.09it/s]\u001b[A\n",
      "97121it [32:10, 52.06it/s]\u001b[A\n",
      "97153it [32:11, 52.12it/s]\u001b[A\n",
      "97185it [32:12, 52.10it/s]\u001b[A\n",
      "97217it [32:12, 52.10it/s]\u001b[A\n",
      "97249it [32:13, 52.11it/s]\u001b[A\n",
      "97281it [32:13, 52.09it/s]\u001b[A\n",
      "97313it [32:14, 52.09it/s]\u001b[A\n",
      "97345it [32:15, 52.03it/s]\u001b[A\n",
      "97377it [32:15, 52.13it/s]\u001b[A\n",
      "97409it [32:16, 52.07it/s]\u001b[A\n",
      "97441it [32:17, 52.12it/s]\u001b[A\n",
      "97473it [32:17, 52.11it/s]\u001b[A\n",
      "97505it [32:18, 52.11it/s]\u001b[A\n",
      "97537it [32:18, 52.10it/s]\u001b[A\n",
      "97569it [32:19, 52.11it/s]\u001b[A\n",
      "97601it [32:20, 52.09it/s]\u001b[A\n",
      "97633it [32:20, 52.08it/s]\u001b[A\n",
      "97665it [32:21, 52.07it/s]\u001b[A\n",
      "97697it [32:21, 52.12it/s]\u001b[A\n",
      "97729it [32:22, 52.14it/s]\u001b[A\n",
      "97761it [32:23, 52.03it/s]\u001b[A\n",
      "97793it [32:23, 52.14it/s]\u001b[A\n",
      "97825it [32:24, 52.14it/s]\u001b[A\n",
      "97857it [32:25, 51.96it/s]\u001b[A\n",
      "97889it [32:25, 52.16it/s]\u001b[A\n",
      "97921it [32:26, 51.80it/s]\u001b[A\n",
      "97953it [32:26, 51.76it/s]\u001b[A\n",
      "97985it [32:27, 51.75it/s]\u001b[A\n",
      "98017it [32:28, 51.88it/s]\u001b[A\n",
      "98049it [32:28, 51.97it/s]\u001b[A\n",
      "98081it [32:29, 51.99it/s]\u001b[A\n",
      "98113it [32:29, 52.07it/s]\u001b[A\n",
      "98145it [32:30, 52.07it/s]\u001b[A\n",
      "98177it [32:31, 52.03it/s]\u001b[A\n",
      "98209it [32:31, 52.11it/s]\u001b[A\n",
      "98241it [32:32, 52.08it/s]\u001b[A\n",
      "98273it [32:33, 52.14it/s]\u001b[A\n",
      "98305it [32:33, 52.06it/s]\u001b[A\n",
      "98337it [32:34, 52.05it/s]\u001b[A\n",
      "98369it [32:34, 52.10it/s]\u001b[A\n",
      "98401it [32:35, 52.13it/s]\u001b[A\n",
      "98433it [32:36, 52.21it/s]\u001b[A\n",
      "98465it [32:36, 52.15it/s]\u001b[A\n",
      "98497it [32:37, 52.04it/s]\u001b[A\n",
      "98529it [32:37, 52.02it/s]\u001b[A\n",
      "98561it [32:38, 52.13it/s]\u001b[A\n",
      "98593it [32:39, 52.10it/s]\u001b[A\n",
      "98625it [32:39, 52.12it/s]\u001b[A\n",
      "98657it [32:40, 52.14it/s]\u001b[A\n",
      "98689it [32:41, 52.14it/s]\u001b[A\n",
      "98721it [32:41, 52.12it/s]\u001b[A\n",
      "98753it [32:42, 52.12it/s]\u001b[A\n",
      "98785it [32:42, 52.11it/s]\u001b[A\n",
      "98817it [32:43, 52.17it/s]\u001b[A\n",
      "98849it [32:44, 52.11it/s]\u001b[A\n",
      "98881it [32:44, 52.14it/s]\u001b[A\n",
      "98913it [32:45, 52.09it/s]\u001b[A\n",
      "98945it [32:45, 52.13it/s]\u001b[A\n",
      "98977it [32:46, 52.16it/s]\u001b[A\n",
      "99009it [32:47, 51.72it/s]\u001b[A\n",
      "99041it [32:47, 51.60it/s]\u001b[A\n",
      "99073it [32:48, 51.87it/s]\u001b[A\n",
      "99105it [32:49, 51.90it/s]\u001b[A\n",
      "99137it [32:49, 51.99it/s]\u001b[A\n",
      "99169it [32:50, 52.10it/s]\u001b[A\n",
      "99201it [32:50, 52.05it/s]\u001b[A\n",
      "99233it [32:51, 51.96it/s]\u001b[A\n",
      "99265it [32:52, 52.09it/s]\u001b[A\n",
      "99297it [32:52, 52.15it/s]\u001b[A\n",
      "99329it [32:53, 52.13it/s]\u001b[A\n",
      "99361it [32:53, 51.94it/s]\u001b[A\n",
      "99393it [32:54, 52.15it/s]\u001b[A\n",
      "99425it [32:55, 52.19it/s]\u001b[A\n",
      "99457it [32:55, 52.11it/s]\u001b[A\n",
      "99489it [32:56, 52.04it/s]\u001b[A\n",
      "99521it [32:57, 52.10it/s]\u001b[A\n",
      "99553it [32:57, 52.12it/s]\u001b[A\n",
      "99585it [32:58, 52.12it/s]\u001b[A\n",
      "99617it [32:58, 52.13it/s]\u001b[A\n",
      "99649it [32:59, 52.10it/s]\u001b[A\n",
      "99681it [33:00, 52.11it/s]\u001b[A\n",
      "99713it [33:00, 52.05it/s]\u001b[A\n",
      "99745it [33:01, 52.12it/s]\u001b[A\n",
      "99777it [33:01, 52.12it/s]\u001b[A\n",
      "99809it [33:02, 52.16it/s]\u001b[A\n",
      "99841it [33:03, 52.14it/s]\u001b[A\n",
      "99873it [33:03, 52.11it/s]\u001b[A\n",
      "99905it [33:04, 52.07it/s]\u001b[A\n",
      "99937it [33:04, 52.12it/s]\u001b[A\n",
      "99969it [33:05, 52.07it/s]\u001b[A\n",
      "100001it [33:06, 52.13it/s]\u001b[A\n",
      "100033it [33:06, 52.12it/s]\u001b[A\n",
      "100065it [33:07, 52.07it/s]\u001b[A\n",
      "100097it [33:08, 52.07it/s]\u001b[A\n",
      "100129it [33:08, 52.11it/s]\u001b[A\n",
      "100161it [33:09, 52.09it/s]\u001b[A\n",
      "100193it [33:09, 52.10it/s]\u001b[A\n",
      "100225it [33:10, 52.12it/s]\u001b[A\n",
      "100257it [33:11, 52.06it/s]\u001b[A\n",
      "100289it [33:11, 52.10it/s]\u001b[A\n",
      "100321it [33:12, 52.16it/s]\u001b[A\n",
      "100353it [33:12, 52.15it/s]\u001b[A\n",
      "100385it [33:13, 52.05it/s]\u001b[A\n",
      "100417it [33:14, 52.12it/s]\u001b[A\n",
      "100449it [33:14, 52.13it/s]\u001b[A\n",
      "100481it [33:15, 52.17it/s]\u001b[A\n",
      "100513it [33:16, 52.15it/s]\u001b[A\n",
      "100545it [33:16, 52.12it/s]\u001b[A\n",
      "100577it [33:17, 52.16it/s]\u001b[A\n",
      "100609it [33:17, 52.10it/s]\u001b[A\n",
      "100641it [33:18, 52.20it/s]\u001b[A\n",
      "100673it [33:19, 52.16it/s]\u001b[A\n",
      "100705it [33:19, 52.09it/s]\u001b[A\n",
      "100737it [33:20, 52.13it/s]\u001b[A\n",
      "100769it [33:20, 52.03it/s]\u001b[A\n",
      "100801it [33:21, 52.09it/s]\u001b[A\n",
      "100833it [33:22, 52.15it/s]\u001b[A\n",
      "100865it [33:22, 52.09it/s]\u001b[A\n",
      "100897it [33:23, 52.12it/s]\u001b[A\n",
      "100929it [33:24, 52.17it/s]\u001b[A\n",
      "100961it [33:24, 52.14it/s]\u001b[A\n",
      "100993it [33:25, 52.09it/s]\u001b[A\n",
      "101025it [33:25, 52.12it/s]\u001b[A\n",
      "101057it [33:26, 52.12it/s]\u001b[A\n",
      "101089it [33:27, 52.13it/s]\u001b[A\n",
      "101121it [33:27, 52.11it/s]\u001b[A\n",
      "101153it [33:28, 52.08it/s]\u001b[A\n",
      "101185it [33:28, 52.15it/s]\u001b[A\n",
      "101217it [33:29, 52.11it/s]\u001b[A\n",
      "101249it [33:30, 52.15it/s]\u001b[A\n",
      "101281it [33:30, 52.11it/s]\u001b[A\n",
      "101313it [33:31, 52.06it/s]\u001b[A\n",
      "101345it [33:32, 52.15it/s]\u001b[A\n",
      "101377it [33:32, 52.14it/s]\u001b[A\n",
      "101409it [33:33, 52.13it/s]\u001b[A\n",
      "101441it [33:33, 52.09it/s]\u001b[A\n",
      "101473it [33:34, 52.09it/s]\u001b[A\n",
      "101505it [33:35, 52.24it/s]\u001b[A\n",
      "101537it [33:35, 52.03it/s]\u001b[A\n",
      "101569it [33:36, 52.20it/s]\u001b[A\n",
      "101601it [33:36, 52.02it/s]\u001b[A\n",
      "101633it [33:37, 52.07it/s]\u001b[A\n",
      "101665it [33:38, 52.11it/s]\u001b[A\n",
      "101697it [33:38, 52.09it/s]\u001b[A\n",
      "101729it [33:39, 52.08it/s]\u001b[A\n",
      "101761it [33:39, 52.03it/s]\u001b[A\n",
      "101793it [33:40, 52.06it/s]\u001b[A\n",
      "101825it [33:41, 52.16it/s]\u001b[A\n",
      "101857it [33:41, 52.11it/s]\u001b[A\n",
      "101889it [33:42, 52.05it/s]\u001b[A\n",
      "101921it [33:43, 52.12it/s]\u001b[A\n",
      "101953it [33:43, 52.12it/s]\u001b[A\n",
      "101985it [33:44, 52.12it/s]\u001b[A\n",
      "102017it [33:44, 52.14it/s]\u001b[A\n",
      "102049it [33:45, 52.04it/s]\u001b[A\n",
      "102081it [33:46, 52.05it/s]\u001b[A\n",
      "102113it [33:46, 52.05it/s]\u001b[A\n",
      "102145it [33:47, 52.12it/s]\u001b[A\n",
      "102177it [33:47, 52.13it/s]\u001b[A\n",
      "102209it [33:48, 52.04it/s]\u001b[A\n",
      "102241it [33:49, 52.10it/s]\u001b[A\n",
      "102273it [33:49, 52.14it/s]\u001b[A\n",
      "102305it [33:50, 52.11it/s]\u001b[A\n",
      "102337it [33:51, 52.11it/s]\u001b[A\n",
      "102369it [33:51, 52.06it/s]\u001b[A\n",
      "102401it [33:52, 52.09it/s]\u001b[A\n",
      "102433it [33:52, 52.12it/s]\u001b[A\n",
      "102465it [33:53, 52.02it/s]\u001b[A\n",
      "102497it [33:54, 52.12it/s]\u001b[A\n",
      "102529it [33:54, 52.12it/s]\u001b[A\n",
      "102561it [33:55, 52.10it/s]\u001b[A\n",
      "102593it [33:55, 52.14it/s]\u001b[A\n",
      "102625it [33:56, 52.10it/s]\u001b[A\n",
      "102657it [33:57, 52.03it/s]\u001b[A\n",
      "102689it [33:57, 52.19it/s]\u001b[A\n",
      "102721it [33:58, 52.13it/s]\u001b[A\n",
      "102753it [33:59, 52.12it/s]\u001b[A\n",
      "102785it [33:59, 51.99it/s]\u001b[A\n",
      "102817it [34:00, 52.15it/s]\u001b[A\n",
      "102849it [34:00, 52.16it/s]\u001b[A\n",
      "102881it [34:01, 52.14it/s]\u001b[A\n",
      "102913it [34:02, 52.20it/s]\u001b[A\n",
      "102945it [34:02, 52.08it/s]\u001b[A\n",
      "102977it [34:03, 52.29it/s]\u001b[A\n",
      "103009it [34:03, 52.07it/s]\u001b[A\n",
      "103041it [34:04, 52.03it/s]\u001b[A\n",
      "103073it [34:05, 52.04it/s]\u001b[A\n",
      "103105it [34:05, 52.13it/s]\u001b[A\n",
      "103137it [34:06, 52.11it/s]\u001b[A\n",
      "103169it [34:07, 52.10it/s]\u001b[A\n",
      "103201it [34:07, 52.01it/s]\u001b[A\n",
      "103233it [34:08, 52.08it/s]\u001b[A\n",
      "103265it [34:08, 52.18it/s]\u001b[A\n",
      "103297it [34:09, 52.10it/s]\u001b[A\n",
      "103329it [34:10, 52.23it/s]\u001b[A\n",
      "103361it [34:10, 52.14it/s]\u001b[A\n",
      "103393it [34:11, 52.03it/s]\u001b[A\n",
      "103425it [34:11, 52.14it/s]\u001b[A\n",
      "103457it [34:12, 52.10it/s]\u001b[A\n",
      "103489it [34:13, 52.14it/s]\u001b[A\n",
      "103521it [34:13, 52.10it/s]\u001b[A\n",
      "103553it [34:14, 52.13it/s]\u001b[A\n",
      "103585it [34:15, 52.13it/s]\u001b[A\n",
      "103617it [34:15, 52.07it/s]\u001b[A\n",
      "103649it [34:16, 52.07it/s]\u001b[A\n",
      "103681it [34:16, 52.13it/s]\u001b[A\n",
      "103713it [34:17, 52.19it/s]\u001b[A\n",
      "103745it [34:18, 52.12it/s]\u001b[A\n",
      "103777it [34:18, 52.06it/s]\u001b[A\n",
      "103809it [34:19, 52.17it/s]\u001b[A\n",
      "103841it [34:19, 52.16it/s]\u001b[A\n",
      "103873it [34:20, 52.09it/s]\u001b[A\n",
      "103905it [34:21, 52.13it/s]\u001b[A\n",
      "103937it [34:21, 52.22it/s]\u001b[A\n",
      "103969it [34:22, 51.99it/s]\u001b[A\n",
      "104001it [34:22, 52.04it/s]\u001b[A\n",
      "104033it [34:23, 51.84it/s]\u001b[A\n",
      "104065it [34:24, 52.09it/s]\u001b[A\n",
      "104097it [34:24, 52.07it/s]\u001b[A\n",
      "104129it [34:25, 52.10it/s]\u001b[A\n",
      "104161it [34:26, 52.09it/s]\u001b[A\n",
      "104193it [34:26, 52.15it/s]\u001b[A\n",
      "104225it [34:27, 52.12it/s]\u001b[A\n",
      "104257it [34:27, 52.12it/s]\u001b[A\n",
      "104289it [34:28, 52.01it/s]\u001b[A\n",
      "104321it [34:29, 52.06it/s]\u001b[A\n",
      "104353it [34:29, 52.15it/s]\u001b[A\n",
      "104385it [34:30, 52.14it/s]\u001b[A\n",
      "104417it [34:30, 52.08it/s]\u001b[A\n",
      "104449it [34:31, 52.15it/s]\u001b[A\n",
      "104481it [34:32, 51.83it/s]\u001b[A\n",
      "104513it [34:32, 52.07it/s]\u001b[A\n",
      "104545it [34:33, 52.12it/s]\u001b[A\n",
      "104577it [34:34, 52.11it/s]\u001b[A\n",
      "104609it [34:34, 52.19it/s]\u001b[A\n",
      "104641it [34:35, 52.17it/s]\u001b[A\n",
      "104673it [34:35, 52.18it/s]\u001b[A\n",
      "104705it [34:36, 52.15it/s]\u001b[A\n",
      "104737it [34:37, 52.14it/s]\u001b[A\n",
      "104769it [34:37, 52.16it/s]\u001b[A\n",
      "104801it [34:38, 52.09it/s]\u001b[A\n",
      "104833it [34:38, 52.14it/s]\u001b[A\n",
      "104865it [34:39, 52.11it/s]\u001b[A\n",
      "104897it [34:40, 52.09it/s]\u001b[A\n",
      "104929it [34:40, 52.14it/s]\u001b[A\n",
      "104961it [34:41, 52.08it/s]\u001b[A\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "104993it [34:42, 52.07it/s]\u001b[A\n",
      "105025it [34:42, 52.13it/s]\u001b[A\n",
      "105057it [34:43, 51.99it/s]\u001b[A\n",
      "105089it [34:43, 52.01it/s]\u001b[A\n",
      "105121it [34:44, 52.01it/s]\u001b[A\n",
      "105153it [34:45, 52.11it/s]\u001b[A\n",
      "105185it [34:45, 52.11it/s]\u001b[A\n",
      "105217it [34:46, 52.19it/s]\u001b[A\n",
      "105249it [34:46, 52.18it/s]\u001b[A\n",
      "105281it [34:47, 52.11it/s]\u001b[A\n",
      "105313it [34:48, 52.19it/s]\u001b[A\n",
      "105345it [34:48, 52.09it/s]\u001b[A\n",
      "105377it [34:49, 52.19it/s]\u001b[A\n",
      "105409it [34:50, 52.21it/s]\u001b[A\n",
      "105441it [34:50, 52.12it/s]\u001b[A\n",
      "105473it [34:51, 52.12it/s]\u001b[A\n",
      "105505it [34:51, 52.12it/s]\u001b[A\n",
      "105537it [34:52, 51.86it/s]\u001b[A\n",
      "105569it [34:53, 52.21it/s]\u001b[A\n",
      "105601it [34:53, 52.17it/s]\u001b[A\n",
      "105633it [34:54, 52.09it/s]\u001b[A\n",
      "105665it [34:54, 52.13it/s]\u001b[A\n",
      "105697it [34:55, 52.14it/s]\u001b[A\n",
      "105729it [34:56, 52.14it/s]\u001b[A\n",
      "105761it [34:56, 52.11it/s]\u001b[A\n",
      "105793it [34:57, 52.20it/s]\u001b[A\n",
      "105825it [34:57, 52.07it/s]\u001b[A\n",
      "105857it [34:58, 52.10it/s]\u001b[A\n",
      "105889it [34:59, 52.09it/s]\u001b[A\n",
      "105921it [34:59, 52.13it/s]\u001b[A\n",
      "105953it [35:00, 52.10it/s]\u001b[A\n",
      "105985it [35:01, 52.13it/s]\u001b[A\n",
      "106017it [35:01, 52.06it/s]\u001b[A\n",
      "106049it [35:02, 52.02it/s]\u001b[A\n",
      "106081it [35:02, 52.11it/s]\u001b[A\n",
      "106113it [35:03, 52.12it/s]\u001b[A\n",
      "106145it [35:04, 52.17it/s]\u001b[A\n",
      "106177it [35:04, 52.08it/s]\u001b[A\n",
      "106209it [35:05, 52.09it/s]\u001b[A\n",
      "106241it [35:05, 52.06it/s]\u001b[A\n",
      "106273it [35:06, 52.11it/s]\u001b[A\n",
      "106305it [35:07, 52.10it/s]\u001b[A\n",
      "106337it [35:07, 52.08it/s]\u001b[A\n",
      "106369it [35:08, 52.11it/s]\u001b[A\n",
      "106401it [35:09, 52.06it/s]\u001b[A\n",
      "106433it [35:09, 52.16it/s]\u001b[A\n",
      "106465it [35:10, 52.11it/s]\u001b[A\n",
      "106497it [35:10, 52.11it/s]\u001b[A\n",
      "106529it [35:11, 52.26it/s]\u001b[A\n",
      "106561it [35:12, 52.05it/s]\u001b[A\n",
      "106593it [35:12, 52.13it/s]\u001b[A\n",
      "106625it [35:13, 52.16it/s]\u001b[A\n",
      "106657it [35:13, 52.08it/s]\u001b[A\n",
      "106689it [35:14, 52.13it/s]\u001b[A\n",
      "106721it [35:15, 52.04it/s]\u001b[A\n",
      "106753it [35:15, 52.04it/s]\u001b[A\n",
      "106785it [35:16, 52.12it/s]\u001b[A\n",
      "106817it [35:17, 52.12it/s]\u001b[A\n",
      "106849it [35:17, 52.11it/s]\u001b[A\n",
      "106881it [35:18, 52.23it/s]\u001b[A\n",
      "106913it [35:18, 52.15it/s]\u001b[A\n",
      "106945it [35:19, 52.07it/s]\u001b[A\n",
      "106977it [35:20, 52.04it/s]\u001b[A\n",
      "107009it [35:20, 52.05it/s]\u001b[A\n",
      "107041it [35:21, 52.11it/s]\u001b[A\n",
      "107073it [35:21, 52.04it/s]\u001b[A\n",
      "107105it [35:22, 52.07it/s]\u001b[A\n",
      "107137it [35:23, 52.16it/s]\u001b[A\n",
      "107169it [35:23, 52.09it/s]\u001b[A\n",
      "107201it [35:24, 52.13it/s]\u001b[A\n",
      "107233it [35:25, 52.07it/s]\u001b[A\n",
      "107265it [35:25, 52.12it/s]\u001b[A\n",
      "107297it [35:26, 52.00it/s]\u001b[A\n",
      "107329it [35:26, 52.16it/s]\u001b[A\n",
      "107361it [35:27, 52.12it/s]\u001b[A\n",
      "107393it [35:28, 52.17it/s]\u001b[A\n",
      "107425it [35:28, 52.18it/s]\u001b[A\n",
      "107457it [35:29, 52.16it/s]\u001b[A\n",
      "107489it [35:29, 52.08it/s]\u001b[A\n",
      "107521it [35:30, 52.11it/s]\u001b[A\n",
      "107553it [35:31, 52.14it/s]\u001b[A\n",
      "107585it [35:31, 52.06it/s]\u001b[A\n",
      "107617it [35:32, 52.16it/s]\u001b[A\n",
      "107649it [35:32, 52.10it/s]\u001b[A\n",
      "107681it [35:33, 52.10it/s]\u001b[A\n",
      "107713it [35:34, 52.11it/s]\u001b[A\n",
      "107745it [35:34, 52.03it/s]\u001b[A\n",
      "107777it [35:35, 52.00it/s]\u001b[A\n",
      "107809it [35:36, 52.12it/s]\u001b[A\n",
      "107841it [35:36, 52.07it/s]\u001b[A\n",
      "107873it [35:37, 52.09it/s]\u001b[A\n",
      "107905it [35:37, 52.05it/s]\u001b[A\n",
      "107937it [35:38, 52.12it/s]\u001b[A\n",
      "107969it [35:39, 52.03it/s]\u001b[A\n",
      "108001it [35:39, 52.12it/s]\u001b[A\n",
      "108033it [35:40, 52.04it/s]\u001b[A\n",
      "108065it [35:40, 52.13it/s]\u001b[A\n",
      "108097it [35:41, 52.08it/s]\u001b[A\n",
      "108129it [35:42, 52.10it/s]\u001b[A\n",
      "108161it [35:42, 52.15it/s]\u001b[A\n",
      "108193it [35:43, 52.11it/s]\u001b[A\n",
      "108225it [35:44, 52.08it/s]\u001b[A\n",
      "108257it [35:44, 52.09it/s]\u001b[A\n",
      "108289it [35:45, 52.14it/s]\u001b[A\n",
      "108321it [35:45, 52.13it/s]\u001b[A\n",
      "108353it [35:46, 52.04it/s]\u001b[A\n",
      "108385it [35:47, 52.03it/s]\u001b[A\n",
      "108417it [35:47, 52.07it/s]\u001b[A\n",
      "108449it [35:48, 52.09it/s]\u001b[A\n",
      "108481it [35:48, 52.12it/s]\u001b[A\n",
      "108513it [35:49, 52.20it/s]\u001b[A\n",
      "108545it [35:50, 51.98it/s]\u001b[A\n",
      "108577it [35:50, 52.03it/s]\u001b[A\n",
      "108609it [35:51, 52.02it/s]\u001b[A\n",
      "108641it [35:52, 52.09it/s]\u001b[A\n",
      "108673it [35:52, 52.17it/s]\u001b[A\n",
      "108705it [35:53, 52.09it/s]\u001b[A\n",
      "108737it [35:53, 52.11it/s]\u001b[A\n",
      "108769it [35:54, 52.09it/s]\u001b[A\n",
      "108801it [35:55, 51.98it/s]\u001b[A\n",
      "108833it [35:55, 52.06it/s]\u001b[A\n",
      "108865it [35:56, 52.14it/s]\u001b[A\n",
      "108897it [35:56, 52.14it/s]\u001b[A\n",
      "108929it [35:57, 52.15it/s]\u001b[A\n",
      "108961it [35:58, 52.14it/s]\u001b[A\n",
      "108993it [35:58, 52.06it/s]\u001b[A\n",
      "109025it [35:59, 52.12it/s]\u001b[A\n",
      "109057it [36:00, 52.13it/s]\u001b[A\n",
      "109089it [36:00, 52.19it/s]\u001b[A\n",
      "109121it [36:01, 52.13it/s]\u001b[A\n",
      "109153it [36:01, 52.09it/s]\u001b[A\n",
      "109185it [36:02, 52.13it/s]\u001b[A\n",
      "109217it [36:03, 52.26it/s]\u001b[A\n",
      "109249it [36:03, 52.10it/s]\u001b[A\n",
      "109281it [36:04, 52.09it/s]\u001b[A\n",
      "109313it [36:04, 52.07it/s]\u001b[A\n",
      "109345it [36:05, 52.07it/s]\u001b[A\n",
      "109377it [36:06, 52.14it/s]\u001b[A\n",
      "109409it [36:06, 52.03it/s]\u001b[A\n",
      "109441it [36:07, 52.15it/s]\u001b[A\n",
      "109473it [36:08, 52.16it/s]\u001b[A\n",
      "109505it [36:08, 52.06it/s]\u001b[A\n",
      "109537it [36:09, 52.08it/s]\u001b[A\n",
      "109569it [36:09, 52.04it/s]\u001b[A\n",
      "109601it [36:10, 52.14it/s]\u001b[A\n",
      "109633it [36:11, 52.06it/s]\u001b[A\n",
      "109665it [36:11, 52.13it/s]\u001b[A\n",
      "109697it [36:12, 52.18it/s]\u001b[A\n",
      "109729it [36:12, 52.15it/s]\u001b[A\n",
      "109761it [36:13, 52.16it/s]\u001b[A\n",
      "109793it [36:14, 52.26it/s]\u001b[A\n",
      "109825it [36:14, 52.06it/s]\u001b[A\n",
      "109857it [36:15, 52.07it/s]\u001b[A\n",
      "109889it [36:15, 52.10it/s]\u001b[A\n",
      "109921it [36:16, 52.14it/s]\u001b[A\n",
      "109953it [36:17, 52.16it/s]\u001b[A\n",
      "109985it [36:17, 52.12it/s]\u001b[A\n",
      "110017it [36:18, 52.15it/s]\u001b[A\n",
      "110049it [36:19, 51.94it/s]\u001b[A\n",
      "110081it [36:19, 52.10it/s]\u001b[A\n",
      "110113it [36:20, 52.07it/s]\u001b[A\n",
      "110145it [36:20, 52.12it/s]\u001b[A\n",
      "110177it [36:21, 52.12it/s]\u001b[A\n",
      "110209it [36:22, 52.20it/s]\u001b[A\n",
      "110241it [36:22, 52.14it/s]\u001b[A\n",
      "110273it [36:23, 52.12it/s]\u001b[A\n",
      "110305it [36:23, 52.18it/s]\u001b[A\n",
      "110337it [36:24, 52.14it/s]\u001b[A\n",
      "110369it [36:25, 52.14it/s]\u001b[A\n",
      "110401it [36:25, 52.13it/s]\u001b[A\n",
      "110433it [36:26, 52.09it/s]\u001b[A\n",
      "110465it [36:27, 52.15it/s]\u001b[A\n",
      "110497it [36:27, 52.08it/s]\u001b[A\n",
      "110529it [36:28, 52.16it/s]\u001b[A\n",
      "110561it [36:28, 52.11it/s]\u001b[A\n",
      "110593it [36:29, 52.07it/s]\u001b[A\n",
      "110625it [36:30, 52.09it/s]\u001b[A\n",
      "110657it [36:30, 52.15it/s]\u001b[A\n",
      "110689it [36:31, 52.13it/s]\u001b[A\n",
      "110721it [36:31, 52.14it/s]\u001b[A\n",
      "110753it [36:32, 52.16it/s]\u001b[A\n",
      "110785it [36:33, 52.08it/s]\u001b[A\n",
      "110817it [36:33, 52.07it/s]\u001b[A\n",
      "110849it [36:34, 52.12it/s]\u001b[A\n",
      "110881it [36:35, 52.16it/s]\u001b[A\n",
      "110913it [36:35, 52.09it/s]\u001b[A\n",
      "110945it [36:36, 52.12it/s]\u001b[A\n",
      "110977it [36:36, 52.04it/s]\u001b[A\n",
      "111009it [36:37, 52.12it/s]\u001b[A\n",
      "111041it [36:38, 52.12it/s]\u001b[A\n",
      "111073it [36:38, 52.08it/s]\u001b[A\n",
      "111105it [36:39, 52.07it/s]\u001b[A\n",
      "111137it [36:39, 52.17it/s]\u001b[A\n",
      "111169it [36:40, 52.08it/s]\u001b[A\n",
      "111201it [36:41, 52.12it/s]\u001b[A\n",
      "111233it [36:41, 52.13it/s]\u001b[A\n",
      "111265it [36:42, 52.12it/s]\u001b[A\n",
      "111297it [36:43, 52.11it/s]\u001b[A\n",
      "111329it [36:43, 52.15it/s]\u001b[A\n",
      "111361it [36:44, 52.12it/s]\u001b[A\n",
      "111393it [36:44, 52.12it/s]\u001b[A\n",
      "111425it [36:45, 52.09it/s]\u001b[A\n",
      "111457it [36:46, 52.07it/s]\u001b[A\n",
      "111489it [36:46, 52.13it/s]\u001b[A\n",
      "111521it [36:47, 52.12it/s]\u001b[A\n",
      "111553it [36:47, 52.10it/s]\u001b[A\n",
      "111585it [36:48, 52.12it/s]\u001b[A\n",
      "111617it [36:49, 52.14it/s]\u001b[A\n",
      "111649it [36:49, 52.11it/s]\u001b[A\n",
      "111681it [36:50, 52.14it/s]\u001b[A\n",
      "111713it [36:50, 52.08it/s]\u001b[A\n",
      "111745it [36:51, 52.10it/s]\u001b[A\n",
      "111777it [36:52, 52.10it/s]\u001b[A\n",
      "111809it [36:52, 52.14it/s]\u001b[A\n",
      "111841it [36:53, 52.25it/s]\u001b[A\n",
      "111873it [36:54, 52.10it/s]\u001b[A\n",
      "111905it [36:54, 52.08it/s]\u001b[A\n",
      "111937it [36:55, 52.08it/s]\u001b[A\n",
      "111969it [36:55, 52.14it/s]\u001b[A\n",
      "112001it [36:56, 52.05it/s]\u001b[A\n",
      "112033it [36:57, 52.12it/s]\u001b[A\n",
      "112065it [36:57, 52.11it/s]\u001b[A\n",
      "112097it [36:58, 52.05it/s]\u001b[A\n",
      "112129it [36:58, 52.09it/s]\u001b[A\n",
      "112161it [36:59, 52.06it/s]\u001b[A\n",
      "112193it [37:00, 52.10it/s]\u001b[A\n",
      "112225it [37:00, 51.95it/s]\u001b[A\n",
      "112257it [37:01, 52.13it/s]\u001b[A\n",
      "112289it [37:02, 52.13it/s]\u001b[A\n",
      "112321it [37:02, 52.15it/s]\u001b[A\n",
      "112353it [37:03, 52.10it/s]\u001b[A\n",
      "112385it [37:03, 52.10it/s]\u001b[A\n",
      "112417it [37:04, 52.11it/s]\u001b[A\n",
      "112449it [37:05, 52.16it/s]\u001b[A\n",
      "112481it [37:05, 52.08it/s]\u001b[A\n",
      "112513it [37:06, 52.14it/s]\u001b[A\n",
      "112545it [37:06, 52.17it/s]\u001b[A\n",
      "112577it [37:07, 52.11it/s]\u001b[A\n",
      "112609it [37:08, 52.09it/s]\u001b[A\n",
      "112641it [37:08, 52.08it/s]\u001b[A\n",
      "112673it [37:09, 52.11it/s]\u001b[A\n",
      "112705it [37:10, 52.21it/s]\u001b[A\n",
      "112737it [37:10, 52.16it/s]\u001b[A\n",
      "112769it [37:11, 52.10it/s]\u001b[A\n",
      "112801it [37:11, 52.17it/s]\u001b[A\n",
      "112833it [37:12, 52.10it/s]\u001b[A\n",
      "112865it [37:13, 52.09it/s]\u001b[A\n",
      "112897it [37:13, 52.08it/s]\u001b[A\n",
      "112929it [37:14, 52.11it/s]\u001b[A\n",
      "112961it [37:14, 52.05it/s]\u001b[A\n",
      "112993it [37:15, 52.10it/s]\u001b[A\n",
      "113025it [37:16, 52.15it/s]\u001b[A\n",
      "113057it [37:16, 51.86it/s]\u001b[A\n",
      "113089it [37:17, 52.19it/s]\u001b[A\n",
      "113121it [37:18, 52.28it/s]\u001b[A\n",
      "113153it [37:18, 52.09it/s]\u001b[A\n",
      "113185it [37:19, 52.08it/s]\u001b[A\n",
      "113217it [37:19, 52.08it/s]\u001b[A\n",
      "113249it [37:20, 52.07it/s]\u001b[A\n",
      "113281it [37:21, 52.11it/s]\u001b[A\n",
      "113313it [37:21, 52.17it/s]\u001b[A\n",
      "113345it [37:22, 52.10it/s]\u001b[A\n",
      "113377it [37:22, 52.15it/s]\u001b[A\n",
      "113409it [37:23, 52.05it/s]\u001b[A\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "113441it [37:24, 52.11it/s]\u001b[A\n",
      "113473it [37:24, 51.99it/s]\u001b[A\n",
      "113505it [37:25, 52.19it/s]\u001b[A\n",
      "113537it [37:25, 52.14it/s]\u001b[A\n",
      "113569it [37:26, 52.10it/s]\u001b[A\n",
      "113601it [37:27, 52.13it/s]\u001b[A\n",
      "113633it [37:27, 52.19it/s]\u001b[A\n",
      "113665it [37:28, 52.14it/s]\u001b[A\n",
      "113697it [37:29, 52.16it/s]\u001b[A\n",
      "113729it [37:29, 52.15it/s]\u001b[A\n",
      "113761it [37:30, 52.12it/s]\u001b[A\n",
      "113793it [37:30, 52.13it/s]\u001b[A\n",
      "113825it [37:31, 52.11it/s]\u001b[A\n",
      "113857it [37:32, 52.08it/s]\u001b[A\n",
      "113889it [37:32, 52.08it/s]\u001b[A\n",
      "113921it [37:33, 52.11it/s]\u001b[A\n",
      "113953it [37:33, 52.17it/s]\u001b[A\n",
      "113985it [37:34, 52.05it/s]\u001b[A\n",
      "114017it [37:35, 52.19it/s]\u001b[A\n",
      "114049it [37:35, 52.09it/s]\u001b[A\n",
      "114081it [37:36, 52.12it/s]\u001b[A\n",
      "114113it [37:37, 52.09it/s]\u001b[A\n",
      "114145it [37:37, 52.06it/s]\u001b[A\n",
      "114177it [37:38, 52.09it/s]\u001b[A\n",
      "114209it [37:38, 52.15it/s]\u001b[A\n",
      "114241it [37:39, 52.06it/s]\u001b[A\n",
      "114273it [37:40, 52.09it/s]\u001b[A\n",
      "114305it [37:40, 52.09it/s]\u001b[A\n",
      "114337it [37:41, 52.10it/s]\u001b[A\n",
      "114369it [37:41, 52.10it/s]\u001b[A\n",
      "114401it [37:42, 52.10it/s]\u001b[A\n",
      "114433it [37:43, 52.13it/s]\u001b[A\n",
      "114465it [37:43, 52.13it/s]\u001b[A\n",
      "114497it [37:44, 52.08it/s]\u001b[A\n",
      "114529it [37:45, 52.09it/s]\u001b[A\n",
      "114561it [37:45, 52.18it/s]\u001b[A\n",
      "114593it [37:46, 52.04it/s]\u001b[A\n",
      "114625it [37:46, 52.05it/s]\u001b[A\n",
      "114657it [37:47, 52.14it/s]\u001b[A\n",
      "114689it [37:48, 52.14it/s]\u001b[A\n",
      "114721it [37:48, 52.03it/s]\u001b[A\n",
      "114753it [37:49, 51.78it/s]\u001b[A\n",
      "114785it [37:49, 51.88it/s]\u001b[A\n",
      "114817it [37:50, 51.80it/s]\u001b[A\n",
      "114849it [37:51, 51.90it/s]\u001b[A\n",
      "114881it [37:51, 51.85it/s]\u001b[A\n",
      "114913it [37:52, 51.93it/s]\u001b[A\n",
      "114945it [37:53, 52.08it/s]\u001b[A\n",
      "114977it [37:53, 52.12it/s]\u001b[A\n",
      "115009it [37:54, 52.12it/s]\u001b[A\n",
      "115041it [37:54, 52.00it/s]\u001b[A\n",
      "115073it [37:55, 52.12it/s]\u001b[A\n",
      "115105it [37:56, 52.05it/s]\u001b[A\n",
      "115137it [37:56, 52.11it/s]\u001b[A\n",
      "115169it [37:57, 52.14it/s]\u001b[A\n",
      "115201it [37:57, 52.09it/s]\u001b[A\n",
      "115233it [37:58, 52.09it/s]\u001b[A\n",
      "115265it [37:59, 52.10it/s]\u001b[A\n",
      "115297it [37:59, 52.08it/s]\u001b[A\n",
      "115329it [38:00, 52.06it/s]\u001b[A\n",
      "115361it [38:01, 52.10it/s]\u001b[A\n",
      "115393it [38:01, 52.15it/s]\u001b[A\n",
      "115425it [38:02, 52.13it/s]\u001b[A\n",
      "115457it [38:02, 52.16it/s]\u001b[A\n",
      "115489it [38:03, 52.11it/s]\u001b[A\n",
      "115521it [38:04, 52.10it/s]\u001b[A\n",
      "115553it [38:04, 52.08it/s]\u001b[A\n",
      "115585it [38:05, 52.11it/s]\u001b[A\n",
      "115617it [38:05, 52.08it/s]\u001b[A\n",
      "115649it [38:06, 52.14it/s]\u001b[A\n",
      "115681it [38:07, 52.11it/s]\u001b[A\n",
      "115713it [38:07, 52.08it/s]\u001b[A\n",
      "115745it [38:08, 52.12it/s]\u001b[A\n",
      "115777it [38:08, 52.11it/s]\u001b[A\n",
      "115809it [38:09, 52.13it/s]\u001b[A\n",
      "115841it [38:10, 52.09it/s]\u001b[A\n",
      "115873it [38:10, 52.16it/s]\u001b[A\n",
      "115905it [38:11, 52.18it/s]\u001b[A\n",
      "115937it [38:12, 52.07it/s]\u001b[A\n",
      "115969it [38:12, 52.09it/s]\u001b[A\n",
      "116001it [38:13, 52.14it/s]\u001b[A\n",
      "116033it [38:13, 52.18it/s]\u001b[A\n",
      "116065it [38:14, 52.10it/s]\u001b[A\n",
      "116097it [38:15, 52.19it/s]\u001b[A\n",
      "116129it [38:15, 52.11it/s]\u001b[A\n",
      "116161it [38:16, 52.14it/s]\u001b[A\n",
      "116193it [38:16, 52.09it/s]\u001b[A\n",
      "116225it [38:17, 52.09it/s]\u001b[A\n",
      "116257it [38:18, 52.10it/s]\u001b[A\n",
      "116289it [38:18, 52.09it/s]\u001b[A\n",
      "116321it [38:19, 52.08it/s]\u001b[A\n",
      "116353it [38:20, 52.11it/s]\u001b[A\n",
      "116385it [38:20, 52.03it/s]\u001b[A\n",
      "116417it [38:21, 52.12it/s]\u001b[A\n",
      "116449it [38:21, 52.06it/s]\u001b[A\n",
      "116481it [38:22, 52.11it/s]\u001b[A\n",
      "116513it [38:23, 52.13it/s]\u001b[A\n",
      "116545it [38:23, 52.13it/s]\u001b[A\n",
      "116577it [38:24, 52.12it/s]\u001b[A\n",
      "116609it [38:24, 52.12it/s]\u001b[A\n",
      "116641it [38:25, 52.14it/s]\u001b[A\n",
      "116673it [38:26, 52.19it/s]\u001b[A\n",
      "116705it [38:26, 52.04it/s]\u001b[A\n",
      "116737it [38:27, 52.10it/s]\u001b[A\n",
      "116769it [38:28, 52.11it/s]\u001b[A\n",
      "116801it [38:28, 52.11it/s]\u001b[A\n",
      "116833it [38:29, 52.07it/s]\u001b[A\n",
      "116865it [38:29, 52.08it/s]\u001b[A\n",
      "116897it [38:30, 52.08it/s]\u001b[A\n",
      "116929it [38:31, 52.19it/s]\u001b[A\n",
      "116961it [38:31, 52.10it/s]\u001b[A\n",
      "116993it [38:32, 52.10it/s]\u001b[A\n",
      "117025it [38:32, 52.13it/s]\u001b[A\n",
      "117057it [38:33, 52.13it/s]\u001b[A\n",
      "117089it [38:34, 52.22it/s]\u001b[A\n",
      "117121it [38:34, 52.07it/s]\u001b[A\n",
      "117153it [38:35, 52.09it/s]\u001b[A\n",
      "117185it [38:36, 52.09it/s]\u001b[A\n",
      "117217it [38:36, 52.10it/s]\u001b[A\n",
      "117249it [38:37, 52.10it/s]\u001b[A\n",
      "117281it [38:37, 52.15it/s]\u001b[A\n",
      "117313it [38:38, 52.15it/s]\u001b[A\n",
      "117345it [38:39, 52.24it/s]\u001b[A\n",
      "117377it [38:39, 52.13it/s]\u001b[A\n",
      "117409it [38:40, 52.07it/s]\u001b[A\n",
      "117441it [38:40, 52.06it/s]\u001b[A\n",
      "117473it [38:41, 52.08it/s]\u001b[A\n",
      "117505it [38:42, 52.01it/s]\u001b[A\n",
      "117537it [38:42, 52.03it/s]\u001b[A\n",
      "117569it [38:43, 52.02it/s]\u001b[A\n",
      "117601it [38:44, 52.10it/s]\u001b[A\n",
      "117633it [38:44, 52.12it/s]\u001b[A\n",
      "117665it [38:45, 52.16it/s]\u001b[A\n",
      "117697it [38:45, 52.00it/s]\u001b[A\n",
      "117729it [38:46, 52.09it/s]\u001b[A\n",
      "117761it [38:47, 52.10it/s]\u001b[A\n",
      "117793it [38:47, 52.09it/s]\u001b[A\n",
      "117825it [38:48, 52.07it/s]\u001b[A\n",
      "117857it [38:48, 52.16it/s]\u001b[A\n",
      "117889it [38:49, 52.13it/s]\u001b[A\n",
      "117921it [38:50, 52.15it/s]\u001b[A\n",
      "117953it [38:50, 52.16it/s]\u001b[A\n",
      "117985it [38:51, 52.16it/s]\u001b[A\n",
      "118017it [38:51, 52.11it/s]\u001b[A\n",
      "118049it [38:52, 52.11it/s]\u001b[A\n",
      "118081it [38:53, 52.09it/s]\u001b[A\n",
      "118113it [38:53, 52.05it/s]\u001b[A\n",
      "118145it [38:54, 52.14it/s]\u001b[A\n",
      "118177it [38:55, 52.05it/s]\u001b[A\n",
      "118209it [38:55, 52.07it/s]\u001b[A\n",
      "118241it [38:56, 52.10it/s]\u001b[A\n",
      "118273it [38:56, 52.15it/s]\u001b[A\n",
      "118305it [38:57, 52.09it/s]\u001b[A\n",
      "118337it [38:58, 52.15it/s]\u001b[A\n",
      "118369it [38:58, 51.97it/s]\u001b[A\n",
      "118401it [38:59, 52.10it/s]\u001b[A\n",
      "118433it [38:59, 52.18it/s]\u001b[A\n",
      "118465it [39:00, 52.15it/s]\u001b[A\n",
      "118497it [39:01, 52.18it/s]\u001b[A\n",
      "118529it [39:01, 51.94it/s]\u001b[A\n",
      "118561it [39:02, 52.11it/s]\u001b[A\n",
      "118593it [39:03, 52.17it/s]\u001b[A\n",
      "118625it [39:03, 52.10it/s]\u001b[A\n",
      "118657it [39:04, 52.09it/s]\u001b[A\n",
      "118689it [39:04, 52.05it/s]\u001b[A\n",
      "118721it [39:05, 52.08it/s]\u001b[A\n",
      "118753it [39:06, 52.12it/s]\u001b[A\n",
      "118785it [39:06, 52.06it/s]\u001b[A\n",
      "118817it [39:07, 52.12it/s]\u001b[A\n",
      "118849it [39:07, 52.14it/s]\u001b[A\n",
      "118881it [39:08, 52.14it/s]\u001b[A\n",
      "118913it [39:09, 52.10it/s]\u001b[A\n",
      "118945it [39:09, 52.19it/s]\u001b[A\n",
      "118977it [39:10, 52.08it/s]\u001b[A\n",
      "119009it [39:11, 52.16it/s]\u001b[A\n",
      "119041it [39:11, 52.19it/s]\u001b[A\n",
      "119073it [39:12, 52.17it/s]\u001b[A\n",
      "119105it [39:12, 52.12it/s]\u001b[A\n",
      "119137it [39:13, 52.27it/s]\u001b[A\n",
      "119169it [39:14, 52.12it/s]\u001b[A\n",
      "119201it [39:14, 52.12it/s]\u001b[A\n",
      "119233it [39:15, 52.09it/s]\u001b[A\n",
      "119265it [39:15, 52.14it/s]\u001b[A\n",
      "119297it [39:16, 52.14it/s]\u001b[A\n",
      "119329it [39:17, 52.08it/s]\u001b[A\n",
      "119361it [39:17, 52.09it/s]\u001b[A\n",
      "119393it [39:18, 51.99it/s]\u001b[A\n",
      "119425it [39:19, 52.13it/s]\u001b[A\n",
      "119457it [39:19, 52.12it/s]\u001b[A\n",
      "119489it [39:20, 52.05it/s]\u001b[A\n",
      "119521it [39:20, 52.09it/s]\u001b[A\n",
      "119553it [39:21, 52.08it/s]\u001b[A\n",
      "119585it [39:22, 52.11it/s]\u001b[A\n",
      "119617it [39:22, 52.06it/s]\u001b[A\n",
      "119649it [39:23, 52.07it/s]\u001b[A\n",
      "119681it [39:23, 52.11it/s]\u001b[A\n",
      "119713it [39:24, 52.02it/s]\u001b[A\n",
      "119745it [39:25, 51.98it/s]\u001b[A\n",
      "119777it [39:25, 52.03it/s]\u001b[A\n",
      "119809it [39:26, 52.10it/s]\u001b[A\n",
      "119841it [39:26, 52.08it/s]\u001b[A\n",
      "119873it [39:27, 52.13it/s]\u001b[A\n",
      "119905it [39:28, 52.13it/s]\u001b[A\n",
      "119937it [39:28, 52.12it/s]\u001b[A\n",
      "119969it [39:29, 52.14it/s]\u001b[A\n",
      "120001it [39:30, 52.12it/s]\u001b[A\n",
      "120033it [39:30, 52.15it/s]\u001b[A\n",
      "120065it [39:31, 52.14it/s]\u001b[A\n",
      "120097it [39:31, 52.19it/s]\u001b[A\n",
      "120129it [39:32, 52.07it/s]\u001b[A\n",
      "120161it [39:33, 52.11it/s]\u001b[A\n",
      "120193it [39:33, 52.07it/s]\u001b[A\n",
      "120225it [39:34, 52.06it/s]\u001b[A\n",
      "120257it [39:34, 52.14it/s]\u001b[A\n",
      "120289it [39:35, 52.00it/s]\u001b[A\n",
      "120321it [39:36, 52.15it/s]\u001b[A\n",
      "120353it [39:36, 52.17it/s]\u001b[A\n",
      "120385it [39:37, 52.09it/s]\u001b[A\n",
      "120417it [39:38, 52.11it/s]\u001b[A\n",
      "120449it [39:38, 52.16it/s]\u001b[A\n",
      "120481it [39:39, 52.09it/s]\u001b[A\n",
      "120513it [39:39, 52.12it/s]\u001b[A\n",
      "120545it [39:40, 52.15it/s]\u001b[A\n",
      "120577it [39:41, 52.05it/s]\u001b[A\n",
      "120609it [39:41, 52.08it/s]\u001b[A\n",
      "120641it [39:42, 52.11it/s]\u001b[A\n",
      "120673it [39:42, 52.04it/s]\u001b[A\n",
      "120705it [39:43, 52.09it/s]\u001b[A\n",
      "120737it [39:44, 52.07it/s]\u001b[A\n",
      "120769it [39:44, 52.03it/s]\u001b[A\n",
      "120801it [39:45, 52.09it/s]\u001b[A\n",
      "120833it [39:46, 52.15it/s]\u001b[A\n",
      "120865it [39:46, 52.20it/s]\u001b[A\n",
      "120897it [39:47, 52.06it/s]\u001b[A\n",
      "120929it [39:47, 52.16it/s]\u001b[A\n",
      "120961it [39:48, 52.13it/s]\u001b[A\n",
      "120993it [39:49, 52.13it/s]\u001b[A\n",
      "121025it [39:49, 52.06it/s]\u001b[A\n",
      "121057it [39:50, 52.13it/s]\u001b[A\n",
      "121089it [39:50, 52.11it/s]\u001b[A\n",
      "121121it [39:51, 52.11it/s]\u001b[A\n",
      "121153it [39:52, 52.07it/s]\u001b[A\n",
      "121185it [39:52, 52.07it/s]\u001b[A\n",
      "121217it [39:53, 52.11it/s]\u001b[A\n",
      "121249it [39:54, 52.12it/s]\u001b[A\n",
      "121281it [39:54, 52.14it/s]\u001b[A\n",
      "121313it [39:55, 52.14it/s]\u001b[A\n",
      "121345it [39:55, 52.13it/s]\u001b[A\n",
      "121377it [39:56, 51.94it/s]\u001b[A\n",
      "121409it [39:57, 52.01it/s]\u001b[A\n",
      "121441it [39:57, 52.12it/s]\u001b[A\n",
      "121473it [39:58, 52.15it/s]\u001b[A\n",
      "121505it [39:58, 52.24it/s]\u001b[A\n",
      "121537it [39:59, 52.16it/s]\u001b[A\n",
      "121569it [40:00, 52.18it/s]\u001b[A\n",
      "121601it [40:00, 52.07it/s]\u001b[A\n",
      "121633it [40:01, 51.93it/s]\u001b[A\n",
      "121665it [40:01, 52.13it/s]\u001b[A\n",
      "121697it [40:02, 52.04it/s]\u001b[A\n",
      "121729it [40:03, 52.12it/s]\u001b[A\n",
      "121761it [40:03, 52.08it/s]\u001b[A\n",
      "121793it [40:04, 52.17it/s]\u001b[A\n",
      "121825it [40:05, 52.09it/s]\u001b[A\n",
      "121857it [40:05, 52.17it/s]\u001b[A\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "121889it [40:06, 51.94it/s]\u001b[A\n",
      "121921it [40:06, 52.09it/s]\u001b[A\n",
      "121953it [40:07, 51.98it/s]\u001b[A\n",
      "121985it [40:08, 52.28it/s]\u001b[A\n",
      "122017it [40:08, 52.17it/s]\u001b[A\n",
      "122049it [40:09, 52.16it/s]\u001b[A\n",
      "122081it [40:09, 52.13it/s]\u001b[A\n",
      "122113it [40:10, 52.21it/s]\u001b[A\n",
      "122145it [40:11, 52.05it/s]\u001b[A\n",
      "122177it [40:11, 52.08it/s]\u001b[A\n",
      "122209it [40:12, 52.11it/s]\u001b[A\n",
      "122241it [40:13, 52.09it/s]\u001b[A\n",
      "122273it [40:13, 52.09it/s]\u001b[A\n",
      "122305it [40:14, 52.21it/s]\u001b[A\n",
      "122337it [40:14, 52.17it/s]\u001b[A\n",
      "122369it [40:15, 52.08it/s]\u001b[A\n",
      "122401it [40:16, 52.06it/s]\u001b[A\n",
      "122433it [40:16, 52.08it/s]\u001b[A\n",
      "122465it [40:17, 52.05it/s]\u001b[A\n",
      "122497it [40:17, 52.00it/s]\u001b[A\n",
      "122529it [40:18, 51.97it/s]\u001b[A\n",
      "122561it [40:19, 51.95it/s]\u001b[A\n",
      "122593it [40:19, 52.08it/s]\u001b[A\n",
      "122625it [40:20, 52.10it/s]\u001b[A\n",
      "122657it [40:21, 52.06it/s]\u001b[A\n",
      "122689it [40:21, 52.10it/s]\u001b[A\n",
      "122721it [40:22, 52.07it/s]\u001b[A\n",
      "122753it [40:22, 52.11it/s]\u001b[A\n",
      "122785it [40:23, 51.98it/s]\u001b[A\n",
      "122817it [40:24, 52.03it/s]\u001b[A\n",
      "122849it [40:24, 52.10it/s]\u001b[A\n",
      "122881it [40:25, 52.09it/s]\u001b[A\n",
      "122913it [40:25, 52.13it/s]\u001b[A\n",
      "122945it [40:26, 51.98it/s]\u001b[A\n",
      "122977it [40:27, 51.82it/s]\u001b[A\n",
      "123009it [40:27, 51.79it/s]\u001b[A\n",
      "123041it [40:28, 51.82it/s]\u001b[A\n",
      "123073it [40:29, 51.84it/s]\u001b[A\n",
      "123105it [40:29, 51.98it/s]\u001b[A\n",
      "123137it [40:30, 52.06it/s]\u001b[A\n",
      "123169it [40:30, 52.10it/s]\u001b[A\n",
      "123201it [40:31, 52.05it/s]\u001b[A\n",
      "123233it [40:32, 52.09it/s]\u001b[A\n",
      "123265it [40:32, 52.09it/s]\u001b[A\n",
      "123297it [40:33, 52.12it/s]\u001b[A\n",
      "123329it [40:33, 52.12it/s]\u001b[A\n",
      "123361it [40:34, 52.11it/s]\u001b[A\n",
      "123393it [40:35, 52.14it/s]\u001b[A\n",
      "123425it [40:35, 52.08it/s]\u001b[A\n",
      "123457it [40:36, 52.13it/s]\u001b[A\n",
      "123489it [40:37, 52.15it/s]\u001b[A\n",
      "123521it [40:37, 52.12it/s]\u001b[A\n",
      "123553it [40:38, 51.87it/s]\u001b[A\n",
      "123585it [40:38, 51.71it/s]\u001b[A\n",
      "123617it [40:39, 51.76it/s]\u001b[A\n",
      "123649it [40:40, 51.92it/s]\u001b[A\n",
      "123681it [40:40, 52.07it/s]\u001b[A\n",
      "123713it [40:41, 51.98it/s]\u001b[A\n",
      "123745it [40:41, 52.07it/s]\u001b[A\n",
      "123777it [40:42, 52.06it/s]\u001b[A\n",
      "123809it [40:43, 52.04it/s]\u001b[A\n",
      "123841it [40:43, 52.10it/s]\u001b[A\n",
      "123873it [40:44, 52.17it/s]\u001b[A\n",
      "123905it [40:45, 52.13it/s]\u001b[A\n",
      "123937it [40:45, 52.18it/s]\u001b[A\n",
      "123969it [40:46, 52.08it/s]\u001b[A\n",
      "124001it [40:46, 52.15it/s]\u001b[A\n",
      "124033it [40:47, 52.13it/s]\u001b[A\n",
      "124065it [40:48, 52.14it/s]\u001b[A\n",
      "124097it [40:48, 52.11it/s]\u001b[A\n",
      "124129it [40:49, 52.13it/s]\u001b[A\n",
      "124161it [40:49, 52.06it/s]\u001b[A\n",
      "124193it [40:50, 52.14it/s]\u001b[A\n",
      "124225it [40:51, 52.10it/s]\u001b[A\n",
      "124257it [40:51, 52.14it/s]\u001b[A\n",
      "124289it [40:52, 51.96it/s]\u001b[A\n",
      "124321it [40:53, 52.18it/s]\u001b[A\n",
      "124353it [40:53, 52.15it/s]\u001b[A\n",
      "124385it [40:54, 52.18it/s]\u001b[A\n",
      "124417it [40:54, 52.16it/s]\u001b[A\n",
      "124449it [40:55, 52.16it/s]\u001b[A\n",
      "124481it [40:56, 52.09it/s]\u001b[A\n",
      "124513it [40:56, 52.13it/s]\u001b[A\n",
      "124545it [40:57, 52.09it/s]\u001b[A\n",
      "124577it [40:57, 52.13it/s]\u001b[A\n",
      "124609it [40:58, 52.11it/s]\u001b[A\n",
      "124641it [40:59, 52.03it/s]\u001b[A\n",
      "124673it [40:59, 52.15it/s]\u001b[A\n",
      "124705it [41:00, 52.15it/s]\u001b[A\n",
      "124737it [41:00, 52.12it/s]\u001b[A\n",
      "124769it [41:01, 52.14it/s]\u001b[A\n",
      "124801it [41:02, 52.09it/s]\u001b[A\n",
      "124833it [41:02, 52.14it/s]\u001b[A\n",
      "124865it [41:03, 52.09it/s]\u001b[A\n",
      "124897it [41:04, 52.14it/s]\u001b[A\n",
      "124929it [41:04, 52.14it/s]\u001b[A\n",
      "124961it [41:05, 52.11it/s]\u001b[A\n",
      "124993it [41:05, 52.11it/s]\u001b[A\n",
      "125025it [41:06, 52.20it/s]\u001b[A\n",
      "125057it [41:07, 52.10it/s]\u001b[A\n",
      "125089it [41:07, 52.10it/s]\u001b[A\n",
      "125121it [41:08, 52.12it/s]\u001b[A\n",
      "125153it [41:08, 52.09it/s]\u001b[A\n",
      "125185it [41:09, 52.10it/s]\u001b[A\n",
      "125217it [41:10, 52.05it/s]\u001b[A\n",
      "125249it [41:10, 52.04it/s]\u001b[A\n",
      "125281it [41:11, 52.02it/s]\u001b[A\n",
      "125313it [41:12, 52.10it/s]\u001b[A\n",
      "125345it [41:12, 52.16it/s]\u001b[A\n",
      "125377it [41:13, 52.10it/s]\u001b[A\n",
      "125409it [41:13, 52.04it/s]\u001b[A\n",
      "125441it [41:14, 52.15it/s]\u001b[A\n",
      "125473it [41:15, 52.09it/s]\u001b[A\n",
      "125505it [41:15, 52.09it/s]\u001b[A\n",
      "125537it [41:16, 51.97it/s]\u001b[A\n",
      "125569it [41:16, 52.04it/s]\u001b[A\n",
      "125601it [41:17, 52.14it/s]\u001b[A\n",
      "125633it [41:18, 52.13it/s]\u001b[A\n",
      "125665it [41:18, 52.07it/s]\u001b[A\n",
      "125697it [41:19, 52.09it/s]\u001b[A\n",
      "125729it [41:20, 52.11it/s]\u001b[A\n",
      "125761it [41:20, 52.17it/s]\u001b[A\n",
      "125793it [41:21, 52.12it/s]\u001b[A\n",
      "125825it [41:21, 52.15it/s]\u001b[A\n",
      "125857it [41:22, 52.02it/s]\u001b[A\n",
      "125889it [41:23, 52.05it/s]\u001b[A\n",
      "125921it [41:23, 52.10it/s]\u001b[A\n",
      "125953it [41:24, 52.17it/s]\u001b[A\n",
      "125985it [41:24, 51.99it/s]\u001b[A\n",
      "126017it [41:25, 52.11it/s]\u001b[A\n",
      "126049it [41:26, 52.17it/s]\u001b[A\n",
      "126081it [41:26, 52.12it/s]\u001b[A\n",
      "126113it [41:27, 52.15it/s]\u001b[A\n",
      "126145it [41:28, 52.05it/s]\u001b[A\n",
      "126177it [41:28, 52.14it/s]\u001b[A\n",
      "126209it [41:29, 52.22it/s]\u001b[A\n",
      "126241it [41:29, 52.15it/s]\u001b[A\n",
      "126273it [41:30, 52.11it/s]\u001b[A\n",
      "126305it [41:31, 52.06it/s]\u001b[A\n",
      "126337it [41:31, 52.10it/s]\u001b[A\n",
      "126369it [41:32, 52.11it/s]\u001b[A\n",
      "126401it [41:32, 52.13it/s]\u001b[A\n",
      "126433it [41:33, 52.12it/s]\u001b[A\n",
      "126465it [41:34, 52.07it/s]\u001b[A\n",
      "126497it [41:34, 52.07it/s]\u001b[A\n",
      "126529it [41:35, 52.08it/s]\u001b[A\n",
      "126561it [41:35, 52.15it/s]\u001b[A\n",
      "126593it [41:36, 52.16it/s]\u001b[A\n",
      "126625it [41:37, 52.08it/s]\u001b[A\n",
      "126657it [41:37, 52.07it/s]\u001b[A\n",
      "126689it [41:38, 52.12it/s]\u001b[A\n",
      "126721it [41:39, 52.07it/s]\u001b[A\n",
      "126753it [41:39, 52.10it/s]\u001b[A\n",
      "126785it [41:40, 51.91it/s]\u001b[A\n",
      "126817it [41:40, 51.99it/s]\u001b[A\n",
      "126849it [41:41, 52.20it/s]\u001b[A\n",
      "126881it [41:42, 52.18it/s]\u001b[A\n",
      "126913it [41:42, 52.16it/s]\u001b[A\n",
      "126945it [41:43, 52.05it/s]\u001b[A\n",
      "126977it [41:43, 52.08it/s]\u001b[A\n",
      "127009it [41:44, 52.11it/s]\u001b[A\n",
      "127041it [41:45, 52.10it/s]\u001b[A\n",
      "127073it [41:45, 52.10it/s]\u001b[A\n",
      "127105it [41:46, 52.11it/s]\u001b[A\n",
      "127137it [41:47, 52.15it/s]\u001b[A\n",
      "127169it [41:47, 52.16it/s]\u001b[A\n",
      "127201it [41:48, 52.07it/s]\u001b[A\n",
      "127233it [41:48, 52.12it/s]\u001b[A\n",
      "127265it [41:49, 52.17it/s]\u001b[A\n",
      "127297it [41:50, 52.16it/s]\u001b[A\n",
      "127329it [41:50, 52.11it/s]\u001b[A\n",
      "127361it [41:51, 52.11it/s]\u001b[A\n",
      "127393it [41:51, 52.09it/s]\u001b[A\n",
      "127425it [41:52, 52.14it/s]\u001b[A\n",
      "127457it [41:53, 52.02it/s]\u001b[A\n",
      "127489it [41:53, 52.13it/s]\u001b[A\n",
      "127521it [41:54, 52.15it/s]\u001b[A\n",
      "127553it [41:55, 52.12it/s]\u001b[A\n",
      "127585it [41:55, 52.11it/s]\u001b[A\n",
      "127617it [41:56, 52.17it/s]\u001b[A\n",
      "127649it [41:56, 52.08it/s]\u001b[A\n",
      "127681it [41:57, 52.17it/s]\u001b[A\n",
      "127713it [41:58, 52.14it/s]\u001b[A\n",
      "127745it [41:58, 52.12it/s]\u001b[A\n",
      "127777it [41:59, 52.14it/s]\u001b[A\n",
      "127809it [41:59, 51.98it/s]\u001b[A\n",
      "127841it [42:00, 52.18it/s]\u001b[A\n",
      "127873it [42:01, 52.19it/s]\u001b[A\n",
      "127905it [42:01, 52.14it/s]\u001b[A\n",
      "127937it [42:02, 52.16it/s]\u001b[A\n",
      "127969it [42:03, 52.16it/s]\u001b[A\n",
      "128001it [42:03, 52.13it/s]\u001b[A\n",
      "128033it [42:04, 52.18it/s]\u001b[A\n",
      "128065it [42:04, 52.14it/s]\u001b[A\n",
      "128097it [42:05, 52.14it/s]\u001b[A\n",
      "128129it [42:06, 52.10it/s]\u001b[A\n",
      "128161it [42:06, 52.13it/s]\u001b[A\n",
      "128193it [42:07, 52.09it/s]\u001b[A\n",
      "128225it [42:07, 52.08it/s]\u001b[A\n",
      "128257it [42:08, 52.11it/s]\u001b[A\n",
      "128289it [42:09, 52.15it/s]\u001b[A\n",
      "128321it [42:09, 52.02it/s]\u001b[A\n",
      "128353it [42:10, 51.84it/s]\u001b[A\n",
      "128385it [42:11, 51.99it/s]\u001b[A\n",
      "128417it [42:11, 52.03it/s]\u001b[A\n",
      "128449it [42:12, 52.00it/s]\u001b[A\n",
      "128481it [42:12, 52.08it/s]\u001b[A\n",
      "128513it [42:13, 52.13it/s]\u001b[A\n",
      "128545it [42:14, 52.05it/s]\u001b[A\n",
      "128577it [42:14, 52.14it/s]\u001b[A\n",
      "128609it [42:15, 51.96it/s]\u001b[A\n",
      "128641it [42:15, 52.15it/s]\u001b[A\n",
      "128673it [42:16, 52.13it/s]\u001b[A\n",
      "128705it [42:17, 52.09it/s]\u001b[A\n",
      "128737it [42:17, 52.17it/s]\u001b[A\n",
      "128769it [42:18, 52.04it/s]\u001b[A\n",
      "128801it [42:18, 52.10it/s]\u001b[A\n",
      "128833it [42:19, 52.13it/s]\u001b[A\n",
      "128865it [42:20, 52.12it/s]\u001b[A\n",
      "128897it [42:20, 52.13it/s]\u001b[A\n",
      "128929it [42:21, 52.13it/s]\u001b[A\n",
      "128961it [42:22, 52.11it/s]\u001b[A\n",
      "128993it [42:22, 52.12it/s]\u001b[A\n",
      "129025it [42:23, 52.09it/s]\u001b[A\n",
      "129057it [42:23, 52.13it/s]\u001b[A\n",
      "129089it [42:24, 52.11it/s]\u001b[A\n",
      "129121it [42:25, 52.09it/s]\u001b[A\n",
      "129153it [42:25, 52.12it/s]\u001b[A\n",
      "129185it [42:26, 52.05it/s]\u001b[A\n",
      "129217it [42:26, 52.06it/s]\u001b[A\n",
      "129249it [42:27, 52.07it/s]\u001b[A\n",
      "129281it [42:28, 52.08it/s]\u001b[A\n",
      "129313it [42:28, 52.08it/s]\u001b[A\n",
      "129345it [42:29, 52.15it/s]\u001b[A\n",
      "129377it [42:30, 52.06it/s]\u001b[A\n",
      "129409it [42:30, 52.13it/s]\u001b[A\n",
      "129441it [42:31, 52.12it/s]\u001b[A\n",
      "129473it [42:31, 52.20it/s]\u001b[A\n",
      "129505it [42:32, 52.08it/s]\u001b[A\n",
      "129537it [42:33, 52.10it/s]\u001b[A\n",
      "129569it [42:33, 52.13it/s]\u001b[A\n",
      "129601it [42:34, 52.11it/s]\u001b[A\n",
      "129633it [42:34, 51.99it/s]\u001b[A\n",
      "129665it [42:35, 52.04it/s]\u001b[A\n",
      "129697it [42:36, 52.10it/s]\u001b[A\n",
      "129729it [42:36, 52.17it/s]\u001b[A\n",
      "129761it [42:37, 52.16it/s]\u001b[A\n",
      "129793it [42:38, 52.03it/s]\u001b[A\n",
      "129825it [42:38, 52.10it/s]\u001b[A\n",
      "129857it [42:39, 52.17it/s]\u001b[A\n",
      "129889it [42:39, 52.08it/s]\u001b[A\n",
      "129921it [42:40, 52.08it/s]\u001b[A\n",
      "129953it [42:41, 52.05it/s]\u001b[A\n",
      "129985it [42:41, 52.03it/s]\u001b[A\n",
      "130017it [42:42, 52.10it/s]\u001b[A\n",
      "130049it [42:42, 52.04it/s]\u001b[A\n",
      "130081it [42:43, 52.01it/s]\u001b[A\n",
      "130113it [42:44, 52.10it/s]\u001b[A\n",
      "130145it [42:44, 52.16it/s]\u001b[A\n",
      "130177it [42:45, 52.13it/s]\u001b[A\n",
      "130209it [42:46, 52.14it/s]\u001b[A\n",
      "130241it [42:46, 52.21it/s]\u001b[A\n",
      "130273it [42:47, 52.16it/s]\u001b[A\n",
      "130305it [42:47, 52.09it/s]\u001b[A\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "130337it [42:48, 52.10it/s]\u001b[A\n",
      "130369it [42:49, 52.12it/s]\u001b[A\n",
      "130401it [42:49, 52.06it/s]\u001b[A\n",
      "130433it [42:50, 52.12it/s]\u001b[A\n",
      "130465it [42:50, 52.12it/s]\u001b[A\n",
      "130497it [42:51, 52.12it/s]\u001b[A\n",
      "130529it [42:52, 52.14it/s]\u001b[A\n",
      "130561it [42:52, 52.09it/s]\u001b[A\n",
      "130593it [42:53, 57.72it/s]\u001b[A"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:prediction_loop marked as finished\n",
      "INFO:tensorflow:prediction_loop marked as finished\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "130613it [42:53, 50.76it/s]\u001b[A\n",
      "  0%|          | 0/130613 [00:00<?, ?it/s]\u001b[A\n",
      "100%|██████████| 130613/130613 [00:00<00:00, 1964691.43it/s]\u001b[A"
     ]
    }
   ],
   "source": [
    "from tqdm import tqdm\n",
    "preds = []\n",
    "for prediction in tqdm(result):\n",
    "    for class_probability in prediction['probabilities']:\n",
    "        preds.append(float(class_probability))\n",
    "\n",
    "results = []\n",
    "for i in tqdm(range(0,len(preds),2)):\n",
    "    if preds[i] < 0.9:\n",
    "        results.append(1)\n",
    "    else:\n",
    "        results.append(0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "_uuid": "4acc876dc974dd4d93525e3b163859ddfda33b2d"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.96\n",
      "0.71777\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import accuracy_score\n",
    "from sklearn.metrics import f1_score\n",
    "\n",
    "print('%.2f' % accuracy_score(np.array(results), test_labels))\n",
    "print('%.5f' % f1_score(np.array(results), test_labels))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "49299e2de55fc4136689cc43905fbc4690060c65"
   },
   "source": [
    "There are several downsides for BERT at this moment:\n",
    "\n",
    "- Training is expensive. All results on the paper were fine-tuned on a single Cloud TPU, which has 64GB of RAM. It is currently not possible to re-produce most of the BERT-Large results on the paper using a GPU with 12GB - 16GB of RAM, because the maximum batch size that can fit in memory is too small. \n",
    "\n",
    "- At the moment BERT supports only English, though addition of other languages is expected.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "a3cbd0cdf0b747e13c427d07f295905c38c813ec"
   },
   "source": [
    "# Competition test\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "ce1d0b361bfcaf9d58334b0f8facce9944ceadff"
   },
   "source": [
    "We've run a test with all of Quora data on Standard NC6 (6 vcpus, 56 GB memory) and achieved f1 score of 0.71777.(1th place at the end of competition)\n",
    "\n",
    "**You can't use BERT in the competition, the notebook will fail when it comes to real testing.**\n",
    "\n",
    "Training took about 40 hours.\n",
    "Results are really amazing, espetially because it's a raw model with no optimization or ensamble, using the simlest of 3 released models.\n",
    "\n",
    "We didn't even have to preprocess anything, model does it for you.\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
